Let us begin with The features description in both of the datasets we will be working on in this project
● id = client company identifier
● activity_new = category of the company’s activity
● channel_sales = code of the sales channel
● cons_12m = electricity consumption of the past 12 months
● cons_gas_12m = gas consumption of the past 12 months
● cons_last_month = electricity consumption of the last month
● date_activ = date of activation of the contract
● date_end = registered date of the end of the contract
● date_modif_prod = date of the last modification of the product
● date_renewal = date of the next contract renewal
● forecast_cons_12m = forecasted electricity consumption for next 12 months
● forecast_cons_year = forecasted electricity consumption for the next calendar year
● forecast_discount_energy = forecasted value of current discount
● forecast_meter_rent_12m = forecasted bill of meter rental for the next 2 months
● forecast_price_energy_off_peak = forecasted energy price for 1st period (off peak)
● forecast_price_energy_peak = forecasted energy price for 2nd period (peak)
● forecast_price_pow_off_peak = forecasted power price for 1st period (off peak)
● has_gas = indicated if client is also a gas client
● imp_cons = current paid consumption
● margin_gross_pow_ele = gross margin on power subscription
● margin_net_pow_ele = net margin on power subscription
● nb_prod_act = number of active products and services
● net_margin = total net margin
● num_years_antig = antiquity of the client (in number of years)
● origin_up = code of the electricity campaign the customer first subscribed to
● pow_max = subscribed power
● churn = has the client churned over the next 3 months
● id = client company identifier
● price_date = reference date
● price_off_peak_var = price of energy for the 1st period (off peak)
● price_peak_var = price of energy for the 2nd period (peak)
● price_mid_peak_var = price of energy for the 3rd period (mid peak)
● price_off_peak_fix = price of power for the 1st period (off peak)
● price_peak_fix = price of power for the 2nd period (peak)
● price_mid_peak_fix = price of power for the 3rd period (mid peak)
import pandas as pd
import seaborn as sns
import plotly.express as px
import matplotlib.pyplot as plt
import numpy as np
import warnings
from catboost import CatBoostClassifier, Pool
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, r2_score, roc_auc_score, roc_curve, confusion_matrix, plot_confusion_matrix
warnings.filterwarnings('ignore')
import missingno as msno
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from xgboost import XGBClassifier
import lightgbm as lgb
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression, SGDClassifier
import plotly.io as pio
import plotly.express as px
pio.templates.default = "plotly"
pd.set_option('display.max_columns',None)
Let's import our datasets
dfc = pd.read_csv("client_data.csv")
dfp = pd.read_csv("price_data.csv")
dfc.head(50)
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 24011ae4ebbe3035111d65fa7c15bc57 | foosdfpfkusacimwkcsosbicdxkicaua | 0 | 54946 | 0 | 2013-06-15 | 2016-06-15 | 2015-11-01 | 2015-06-23 | 0.00 | 0 | 0.0 | 1.78 | 0.114481 | 0.098142 | 40.606701 | t | 0.00 | 25.44 | 25.44 | 2 | 678.99 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 43.648 | 1 |
| 1 | d29c2c54acc38ff3c0614d0a653813dd | MISSING | 4660 | 0 | 0 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 189.95 | 0 | 0.0 | 16.27 | 0.145711 | 0.000000 | 44.311378 | f | 0.00 | 16.38 | 16.38 | 1 | 18.89 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.800 | 0 |
| 2 | 764c75f661154dac3a6c254cd082ea7d | foosdfpfkusacimwkcsosbicdxkicaua | 544 | 0 | 0 | 2010-04-16 | 2016-04-16 | 2010-04-16 | 2015-04-17 | 47.96 | 0 | 0.0 | 38.72 | 0.165794 | 0.087899 | 44.311378 | f | 0.00 | 28.60 | 28.60 | 1 | 6.60 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.856 | 0 |
| 3 | bba03439a292a1e166f80264c16191cb | lmkebamcaaclubfxadlmueccxoimlema | 1584 | 0 | 0 | 2010-03-30 | 2016-03-30 | 2010-03-30 | 2015-03-31 | 240.04 | 0 | 0.0 | 19.83 | 0.146694 | 0.000000 | 44.311378 | f | 0.00 | 30.22 | 30.22 | 1 | 25.46 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 4 | 149d57cf92fc41cf94415803a877cb4b | MISSING | 4425 | 0 | 526 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 445.75 | 526 | 0.0 | 131.73 | 0.116900 | 0.100015 | 40.606701 | f | 52.32 | 44.91 | 44.91 | 1 | 47.98 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 19.800 | 0 |
| 5 | 1aa498825382410b098937d65c4ec26d | usilxuppasemubllopkaafesmlibmsdf | 8302 | 0 | 1998 | 2011-12-09 | 2016-12-09 | 2015-11-01 | 2015-12-10 | 796.94 | 1998 | 0.0 | 30.12 | 0.164775 | 0.086131 | 45.308378 | f | 181.21 | 33.12 | 33.12 | 1 | 118.89 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 13.200 | 1 |
| 6 | 7ab4bf4878d8f7661dfc20e9b8e18011 | foosdfpfkusacimwkcsosbicdxkicaua | 45097 | 0 | 0 | 2011-12-02 | 2016-12-02 | 2011-12-02 | 2015-12-03 | 8069.28 | 0 | 0.0 | 0.00 | 0.166178 | 0.087538 | 44.311378 | f | 0.00 | 4.04 | 4.04 | 1 | 346.63 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 15.000 | 1 |
| 7 | 01495c955be7ec5e7f3203406785aae0 | foosdfpfkusacimwkcsosbicdxkicaua | 29552 | 0 | 1260 | 2010-04-21 | 2016-04-21 | 2010-04-21 | 2015-04-22 | 864.73 | 751 | 0.0 | 144.49 | 0.115174 | 0.098837 | 40.606701 | f | 70.63 | 53.92 | 53.92 | 1 | 100.09 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 26.400 | 0 |
| 8 | f53a254b1115634330c12c7fdbf7958a | usilxuppasemubllopkaafesmlibmsdf | 2962 | 0 | 0 | 2011-09-23 | 2016-09-23 | 2011-09-23 | 2015-09-25 | 444.38 | 0 | 0.0 | 15.85 | 0.145711 | 0.000000 | 44.311378 | f | 0.00 | 12.82 | 12.82 | 1 | 42.59 | 4 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 9 | 10c1b2f97a2d2a6f10299dc213d1a370 | lmkebamcaaclubfxadlmueccxoimlema | 26064 | 0 | 2188 | 2010-05-04 | 2016-05-04 | 2015-04-29 | 2015-05-05 | 2738.10 | 2188 | 0.0 | 130.43 | 0.115761 | 0.099419 | 40.606701 | f | 219.59 | 33.42 | 33.42 | 1 | 329.60 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 31.500 | 0 |
| 10 | d5e512dbc8118a830a7b8c7f61d89992 | usilxuppasemubllopkaafesmlibmsdf | 10831 | 0 | 3112 | 2011-07-11 | 2016-07-11 | 2011-07-11 | 2015-07-12 | 1204.58 | 3112 | 0.0 | 20.94 | 0.164637 | 0.087381 | 44.311378 | f | 349.48 | 14.52 | 14.52 | 1 | 108.50 | 5 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 11.500 | 0 |
| 11 | 6ee77a019251bcc304c88167e0dfcb4c | foosdfpfkusacimwkcsosbicdxkicaua | 4442 | 0 | 2489 | 2009-11-26 | 2016-11-26 | 2009-11-26 | 2015-11-27 | 561.96 | 2489 | 0.0 | 20.50 | 0.167086 | 0.088444 | 46.305378 | f | 308.41 | 21.09 | 21.09 | 1 | 18.41 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 10.392 | 0 |
| 12 | 2c05848e1a577dd041ea6804bbf5bbf5 | foosdfpfkusacimwkcsosbicdxkicaua | 46121 | 1542867 | 5280 | 2011-09-01 | 2016-10-30 | 2013-10-30 | 2015-11-01 | 4502.43 | 5280 | 0.0 | 145.81 | 0.115237 | 0.100123 | 40.606701 | t | 523.68 | 5.52 | 5.52 | 2 | 533.46 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 16.500 | 0 |
| 13 | 21860c2ff2d5df75503b230ce629c253 | foosdfpfkusacimwkcsosbicdxkicaua | 7726 | 0 | 0 | 2013-01-22 | 2017-01-22 | 2015-11-18 | 2016-01-24 | 914.10 | 0 | 0.0 | 74.66 | 0.162327 | 0.084428 | 44.311378 | f | 0.00 | 33.12 | 33.12 | 1 | 89.02 | 3 | ldkssxwpmemidmecebumciepifcamkci | 13.200 | 0 |
| 14 | c5037389dd8be351d3c40f2973da2391 | MISSING | 9707 | 0 | 0 | 2010-12-01 | 2016-12-01 | 2010-12-01 | 2015-12-02 | 1028.52 | 0 | 0.0 | 15.37 | 0.168242 | 0.089602 | 45.308378 | f | 0.00 | 15.78 | 15.78 | 1 | 111.59 | 5 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.856 | 0 |
| 15 | e4773b60d2e68aac94795782abfc006b | lmkebamcaaclubfxadlmueccxoimlema | 47019 | 0 | 3161 | 2012-04-20 | 2016-04-20 | 2012-04-20 | 2015-04-21 | 1687.29 | 1080 | 0.0 | 16.08 | 0.144902 | 0.000000 | 44.311378 | f | 158.53 | 19.28 | 19.28 | 1 | 130.14 | 4 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
| 16 | bd14bbac3aca710ee5288fcd8511d4bb | MISSING | 786536 | 80583 | 85883 | 2010-02-15 | 2016-02-15 | 2010-02-15 | 2015-02-16 | 13517.50 | 23298 | 0.0 | 16.60 | 0.164058 | 0.086163 | 44.311378 | t | 2585.38 | 25.68 | 25.68 | 2 | 1087.76 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 11.400 | 0 |
| 17 | 4e4a34a6d91ff25d0fdf68733907ad44 | foosdfpfkusacimwkcsosbicdxkicaua | 30776 | 0 | 0 | 2011-08-01 | 2016-07-25 | 2014-07-25 | 2015-07-27 | 3045.16 | 0 | 0.0 | 118.24 | 0.112860 | 0.096521 | 40.606701 | f | 0.00 | 19.29 | 19.29 | 1 | 335.78 | 5 | MISSING | 25.000 | 0 |
| 18 | f2b93783adecf0d0d8d60ab547bda3de | MISSING | 24838 | 0 | 2877 | 2011-05-27 | 2016-05-27 | 2011-05-27 | 2015-05-31 | 1088.35 | 798 | 0.0 | 141.79 | 0.115174 | 0.098837 | 40.606701 | f | 79.36 | 21.57 | 21.57 | 1 | 117.44 | 5 | ldkssxwpmemidmecebumciepifcamkci | 15.001 | 0 |
| 19 | 47e976c5d7f1ee1fee6d60ca39fd0218 | foosdfpfkusacimwkcsosbicdxkicaua | 15411 | 0 | 0 | 2012-05-28 | 2016-05-28 | 2012-05-28 | 2015-05-31 | 194.42 | 0 | 0.0 | 18.31 | 0.145711 | 0.000000 | 44.311378 | f | 0.00 | 23.97 | 23.97 | 1 | 19.07 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 13.000 | 0 |
| 20 | db6ac41fb6972374f28bc6f8e43b9764 | foosdfpfkusacimwkcsosbicdxkicaua | 16782 | 0 | 1908 | 2009-10-01 | 2016-10-01 | 2009-10-01 | 2015-10-03 | 1686.67 | 1908 | 0.0 | 131.77 | 0.115237 | 0.100123 | 40.606701 | t | 189.79 | 6.60 | 6.60 | 2 | 203.35 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 19.800 | 0 |
| 21 | bee0c7e972180fbee50c2ea15b12965c | foosdfpfkusacimwkcsosbicdxkicaua | 26987 | 0 | 4099 | 2011-03-04 | 2016-03-04 | 2011-03-04 | 2015-03-09 | 2761.95 | 4099 | 0.0 | 129.95 | 0.118636 | 0.101750 | 40.606701 | f | 413.68 | 47.35 | 47.35 | 1 | 316.53 | 5 | lxidpiddsbxsbosboudacockeimpuepw | 35.000 | 0 |
| 22 | 78014630fbac3e6aa980361d25cea748 | usilxuppasemubllopkaafesmlibmsdf | 20560 | 0 | 3804 | 2012-12-31 | 2016-12-31 | 2015-11-01 | 2016-01-01 | 3037.82 | 3804 | 0.0 | 0.00 | 0.142417 | 0.000000 | 44.311378 | f | 548.89 | 4.14 | 4.14 | 1 | 255.96 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 11.000 | 0 |
| 23 | 5789c446141a44a9b405db35682f6eb5 | foosdfpfkusacimwkcsosbicdxkicaua | 12157 | 0 | 0 | 2013-02-27 | 2016-02-27 | 2015-12-16 | 2015-03-09 | 1418.81 | 0 | 0.0 | 16.42 | 0.160302 | 0.082403 | 44.311378 | f | 0.00 | 23.40 | 23.40 | 2 | 123.87 | 3 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 10.392 | 0 |
| 24 | ed5bc499fecb8eee257857fd286ebd65 | MISSING | 13122 | 0 | 0 | 2009-12-07 | 2016-12-07 | 2015-11-01 | 2015-12-08 | 1932.42 | 0 | 0.0 | 19.51 | 0.143478 | 0.000000 | 44.311378 | f | 0.00 | 3.56 | 3.56 | 1 | 150.70 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 25 | 389bbbe70248fbeecdf9bb1bd0d1da04 | foosdfpfkusacimwkcsosbicdxkicaua | 3584 | 0 | 322 | 2010-11-17 | 2016-11-17 | 2010-11-17 | 2015-11-18 | 379.83 | 322 | 0.0 | 145.98 | 0.115237 | 0.100123 | 41.105201 | f | 32.61 | 57.72 | 57.72 | 1 | 38.76 | 5 | lxidpiddsbxsbosboudacockeimpuepw | 23.100 | 1 |
| 26 | 31615bef53242dfedeaf275214044ca1 | MISSING | 2632 | 0 | 0 | 2010-07-12 | 2016-07-12 | 2010-07-12 | 2015-07-14 | 394.83 | 0 | 0.0 | 15.98 | 0.145711 | 0.000000 | 44.311378 | f | 0.00 | 14.64 | 14.64 | 1 | 38.62 | 6 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
| 27 | 74ff037708f036de5745ce34d8d9d4df | foosdfpfkusacimwkcsosbicdxkicaua | 4792 | 0 | 0 | 2011-12-12 | 2016-12-12 | 2015-11-01 | 2015-12-14 | 719.40 | 0 | 0.0 | 0.00 | 0.144039 | 0.000000 | 45.806878 | f | 0.00 | 19.42 | 19.42 | 1 | 60.00 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 12.500 | 0 |
| 28 | 7287dad2f1f535d03435ea7c78b557c4 | MISSING | 7950 | 0 | 1100 | 2009-10-01 | 2016-10-01 | 2009-10-01 | 2015-10-02 | 856.81 | 1100 | 0.0 | 132.16 | 0.115237 | 0.100123 | 40.939027 | f | 113.82 | 20.00 | 20.00 | 1 | 94.01 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 15.010 | 0 |
| 29 | 26c7bba7d51f86a16109de505bcd4f52 | lmkebamcaaclubfxadlmueccxoimlema | 1856956 | 41883 | 164355 | 2010-09-20 | 2016-09-20 | 2010-09-20 | 2015-09-23 | 3074.16 | 0 | 0.0 | 99.05 | 0.115237 | 0.100123 | 40.606701 | t | 0.00 | 43.44 | 43.44 | 2 | 487.42 | 5 | ldkssxwpmemidmecebumciepifcamkci | 29.000 | 0 |
| 30 | 44e826a55734d0ca5eeafcae0e991a75 | foosdfpfkusacimwkcsosbicdxkicaua | 324 | 0 | 0 | 2009-07-07 | 2016-07-07 | 2015-05-23 | 2015-07-09 | 8.32 | 0 | 0.0 | 1.08 | 0.143170 | 0.000000 | 44.311378 | f | 0.00 | 18.62 | 18.62 | 2 | 1.27 | 7 | ldkssxwpmemidmecebumciepifcamkci | 13.200 | 1 |
| 31 | 4bffd9d74bfb770ef7763e8e4e88dfc3 | usilxuppasemubllopkaafesmlibmsdf | 2103 | 0 | 0 | 2011-12-14 | 2016-08-06 | 2015-08-06 | 2015-01-05 | 355.17 | 0 | 30.0 | 19.60 | 0.198096 | 0.000000 | 46.305378 | f | 0.00 | 27.96 | 27.96 | 2 | 19.00 | 4 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 10.392 | 0 |
| 32 | 2d9d0546d83c60ed3d5728d744d65c40 | usilxuppasemubllopkaafesmlibmsdf | 19593 | 0 | 0 | 2012-07-03 | 2016-07-03 | 2013-10-31 | 2015-07-04 | 2886.50 | 0 | 0.0 | 17.55 | 0.143166 | 0.000000 | 44.311378 | f | 0.00 | 17.45 | 17.45 | 1 | 243.99 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 13.856 | 0 |
| 33 | 79d29481ee261caf08387bcef9e4346d | usilxuppasemubllopkaafesmlibmsdf | 41814 | 0 | 4159 | 2012-10-10 | 2016-10-10 | 2014-06-26 | 2015-10-11 | 4107.43 | 4159 | 0.0 | 129.97 | 0.116509 | 0.101397 | 40.606701 | f | 412.54 | 10.16 | 10.16 | 1 | 477.42 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 30.450 | 0 |
| 34 | 044d071a5dbc2668cddad4f2b28c91d1 | MISSING | 37969 | 0 | 3407 | 2005-03-28 | 2016-03-28 | 2015-03-09 | 2015-03-29 | 3879.98 | 3407 | 0.0 | 130.68 | 0.114604 | 0.098262 | 40.606701 | f | 330.30 | 44.64 | 44.64 | 1 | 407.98 | 11 | ldkssxwpmemidmecebumciepifcamkci | 33.000 | 0 |
| 35 | af87b64523cf1f4f4a681e2cc44c8629 | foosdfpfkusacimwkcsosbicdxkicaua | 41214 | 0 | 3538 | 2011-11-04 | 2016-11-04 | 2015-11-01 | 2015-11-05 | 4294.49 | 3538 | 0.0 | 146.56 | 0.115360 | 0.100243 | 40.606701 | f | 352.08 | 3.72 | 3.72 | 1 | 505.47 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 19.800 | 0 |
| 36 | 9403f3f9ff06ec9e49db8255ede2b098 | lmkebamcaaclubfxadlmueccxoimlema | 2012791 | 0 | 218619 | 2011-06-22 | 2016-06-22 | 2011-06-22 | 2015-06-23 | 547.35 | 346 | 0.0 | 16.12 | 0.144902 | 0.000000 | 44.311378 | f | 50.79 | 21.52 | 21.52 | 1 | 50.38 | 5 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 37 | 51a7edcf29ed04830da6b3ee79f9e508 | lmkebamcaaclubfxadlmueccxoimlema | 20429 | 0 | 0 | 2010-03-12 | 2016-03-12 | 2010-03-12 | 2015-03-13 | 869.54 | 0 | 0.0 | 120.45 | 0.120372 | 0.103487 | 40.606701 | f | 0.00 | 60.39 | 60.39 | 1 | 111.64 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 26.600 | 0 |
| 38 | d9260de11c54a215fa7363f603bbeaf6 | foosdfpfkusacimwkcsosbicdxkicaua | 5957 | 0 | 794 | 2010-06-08 | 2016-06-08 | 2010-06-08 | 2015-06-23 | 510.87 | 315 | 0.0 | 18.47 | 0.145711 | 0.000000 | 44.311378 | f | 46.17 | 24.40 | 24.40 | 1 | 50.71 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 15.000 | 0 |
| 39 | a853b52e3992aa2df0870fe42d843a65 | usilxuppasemubllopkaafesmlibmsdf | 6047 | 0 | 0 | 2011-11-23 | 2016-11-23 | 2011-11-23 | 2015-11-24 | 913.00 | 0 | 0.0 | 19.54 | 0.147506 | 0.000000 | 45.806878 | f | 0.00 | 17.13 | 17.13 | 1 | 86.43 | 4 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 10.392 | 0 |
| 40 | 91607f6e0ed7143ab1a5b2abc20f1c00 | foosdfpfkusacimwkcsosbicdxkicaua | 1938393 | 0 | 312684 | 2012-09-01 | 2016-09-01 | 2012-09-01 | 2015-09-02 | 3426.48 | 3537 | 0.0 | 51.60 | 0.143166 | 0.000000 | 44.311378 | f | 513.03 | 11.95 | 11.95 | 1 | 279.26 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 13.850 | 0 |
| 41 | 757ea31873662fe77d60e8c4f736a1ec | foosdfpfkusacimwkcsosbicdxkicaua | 9141 | 0 | 1592 | 2012-03-01 | 2016-09-20 | 2015-09-20 | 2014-03-03 | 419.22 | 523 | 0.0 | 16.32 | 0.142881 | 0.000000 | 46.305378 | f | 74.39 | 29.50 | 29.50 | 1 | 30.90 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 13.856 | 0 |
| 42 | d53f9011d82ed1f4850d0d312d04468b | ewpakwlliwisiwduibdlfmalxowmwpci | 4259 | 0 | 0 | 2012-06-25 | 2016-07-09 | 2014-07-09 | 2015-07-12 | 632.17 | 0 | 0.0 | 16.74 | 0.144902 | 0.000000 | 44.311378 | f | 0.00 | 14.64 | 14.64 | 1 | 64.34 | 4 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
| 43 | af47b8ea9a596c200ee39128a95738f9 | lmkebamcaaclubfxadlmueccxoimlema | 12199 | 0 | 0 | 2011-01-10 | 2016-02-29 | 2011-01-10 | 2015-03-09 | 1811.90 | 0 | 0.0 | 21.10 | 0.145877 | 0.000000 | 44.311378 | f | 0.00 | 23.40 | 23.40 | 1 | 156.21 | 5 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
| 44 | f8943edfeb1783f3793d3abc6305c1e3 | usilxuppasemubllopkaafesmlibmsdf | 13859 | 0 | 0 | 2011-03-15 | 2016-03-15 | 2011-03-15 | 2015-03-16 | 8.83 | 0 | 0.0 | 18.26 | 0.146694 | 0.000000 | 44.311378 | f | 0.00 | 30.22 | 30.22 | 1 | 0.88 | 5 | lxidpiddsbxsbosboudacockeimpuepw | 13.200 | 0 |
| 45 | adfec8a906dfa5c6d836231e65825539 | foosdfpfkusacimwkcsosbicdxkicaua | 110534 | 0 | 7546 | 2012-04-04 | 2016-04-04 | 2012-04-04 | 2015-04-05 | 3485.92 | 2797 | 0.0 | 131.87 | 0.116910 | 0.100572 | 40.606701 | f | 272.68 | 32.10 | 32.10 | 1 | 416.74 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 26.610 | 0 |
| 46 | 34f2d4431b7db25cd660b098acf3d3a0 | foosdfpfkusacimwkcsosbicdxkicaua | 79090 | 0 | 3319 | 2009-08-01 | 2016-08-01 | 2015-07-12 | 2015-08-02 | 2690.38 | 2191 | 0.0 | 130.47 | 0.114025 | 0.097683 | 40.606701 | f | 201.40 | 12.36 | 12.36 | 1 | 311.76 | 6 | ldkssxwpmemidmecebumciepifcamkci | 19.800 | 0 |
| 47 | cdcc2c363d5b34232082cee78776e59e | MISSING | 5162 | 0 | 0 | 2010-11-18 | 2016-11-18 | 2010-11-18 | 2015-11-21 | 0.00 | 0 | 0.0 | 0.00 | 0.000600 | 0.000000 | 0.000000 | f | 0.00 | 24.33 | 24.33 | 1 | 80.30 | 5 | lxidpiddsbxsbosboudacockeimpuepw | 13.860 | 0 |
| 48 | b96aa3a8655203318c6b853dbdb0ceb7 | foosdfpfkusacimwkcsosbicdxkicaua | 88178 | 0 | 6094 | 2010-01-15 | 2016-02-14 | 2013-09-14 | 2015-02-16 | 9070.33 | 6094 | 0.0 | 145.41 | 0.115744 | 0.098858 | 40.606701 | f | 611.08 | 32.88 | 32.88 | 1 | 719.37 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 33.000 | 1 |
| 49 | 54a77b7ec412ca7133750d2cb2a24096 | ewpakwlliwisiwduibdlfmalxowmwpci | 22333 | 0 | 2237 | 2010-11-17 | 2016-11-17 | 2015-05-24 | 2015-11-18 | 2377.86 | 2237 | 0.0 | 143.04 | 0.115237 | 0.100123 | 40.606701 | f | 226.86 | 9.93 | 9.93 | 1 | 234.39 | 5 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 52.500 | 0 |
dfp.head()
| id | price_date | price_off_peak_var | price_peak_var | price_mid_peak_var | price_off_peak_fix | price_peak_fix | price_mid_peak_fix | |
|---|---|---|---|---|---|---|---|---|
| 0 | 038af19179925da21a25619c5a24b745 | 2015-01-01 | 0.151367 | 0.0 | 0.0 | 44.266931 | 0.0 | 0.0 |
| 1 | 038af19179925da21a25619c5a24b745 | 2015-02-01 | 0.151367 | 0.0 | 0.0 | 44.266931 | 0.0 | 0.0 |
| 2 | 038af19179925da21a25619c5a24b745 | 2015-03-01 | 0.151367 | 0.0 | 0.0 | 44.266931 | 0.0 | 0.0 |
| 3 | 038af19179925da21a25619c5a24b745 | 2015-04-01 | 0.149626 | 0.0 | 0.0 | 44.266931 | 0.0 | 0.0 |
| 4 | 038af19179925da21a25619c5a24b745 | 2015-05-01 | 0.149626 | 0.0 | 0.0 | 44.266931 | 0.0 | 0.0 |
dfc.shape
(14606, 26)
dfp.shape
(193002, 8)
Our clients dataset has 14606 rows and 26 features whereas our prices dataset has 193002 rows and 8 features
dfc.duplicated(['id']).sum()
0
dfp.duplicated(["id"]).sum()
176906
len(dfc['id'].unique())
14606
len(dfp['id'].unique())
16096
dfc.describe()
| cons_12m | cons_gas_12m | cons_last_month | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 1.460600e+04 | 1.460600e+04 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 | 14606.000000 |
| mean | 1.592203e+05 | 2.809238e+04 | 16090.269752 | 1868.614880 | 1399.762906 | 0.966726 | 63.086871 | 0.137283 | 0.050491 | 43.130056 | 152.786896 | 24.565121 | 24.562517 | 1.292346 | 189.264522 | 4.997809 | 18.135136 | 0.097152 |
| std | 5.734653e+05 | 1.629731e+05 | 64364.196422 | 2387.571531 | 3247.786255 | 5.108289 | 66.165783 | 0.024623 | 0.049037 | 4.485988 | 341.369366 | 20.231172 | 20.230280 | 0.709774 | 311.798130 | 1.611749 | 13.534743 | 0.296175 |
| min | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 3.300000 | 0.000000 |
| 25% | 5.674750e+03 | 0.000000e+00 | 0.000000 | 494.995000 | 0.000000 | 0.000000 | 16.180000 | 0.116340 | 0.000000 | 40.606701 | 0.000000 | 14.280000 | 14.280000 | 1.000000 | 50.712500 | 4.000000 | 12.500000 | 0.000000 |
| 50% | 1.411550e+04 | 0.000000e+00 | 792.500000 | 1112.875000 | 314.000000 | 0.000000 | 18.795000 | 0.143166 | 0.084138 | 44.311378 | 37.395000 | 21.640000 | 21.640000 | 1.000000 | 112.530000 | 5.000000 | 13.856000 | 0.000000 |
| 75% | 4.076375e+04 | 0.000000e+00 | 3383.000000 | 2401.790000 | 1745.750000 | 0.000000 | 131.030000 | 0.146348 | 0.098837 | 44.311378 | 193.980000 | 29.880000 | 29.880000 | 1.000000 | 243.097500 | 6.000000 | 19.172500 | 0.000000 |
| max | 6.207104e+06 | 4.154590e+06 | 771203.000000 | 82902.830000 | 175375.000000 | 30.000000 | 599.310000 | 0.273963 | 0.195975 | 59.266378 | 15042.790000 | 374.640000 | 374.640000 | 32.000000 | 24570.650000 | 13.000000 | 320.000000 | 1.000000 |
dfp.describe()
| price_off_peak_var | price_peak_var | price_mid_peak_var | price_off_peak_fix | price_peak_fix | price_mid_peak_fix | |
|---|---|---|---|---|---|---|
| count | 193002.000000 | 193002.000000 | 193002.000000 | 193002.000000 | 193002.000000 | 193002.000000 |
| mean | 0.141027 | 0.054630 | 0.030496 | 43.334477 | 10.622875 | 6.409984 |
| std | 0.025032 | 0.049924 | 0.036298 | 5.410297 | 12.841895 | 7.773592 |
| min | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 25% | 0.125976 | 0.000000 | 0.000000 | 40.728885 | 0.000000 | 0.000000 |
| 50% | 0.146033 | 0.085483 | 0.000000 | 44.266930 | 0.000000 | 0.000000 |
| 75% | 0.151635 | 0.101673 | 0.072558 | 44.444710 | 24.339581 | 16.226389 |
| max | 0.280700 | 0.229788 | 0.114102 | 59.444710 | 36.490692 | 17.458221 |
dfc.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 14606 entries, 0 to 14605 Data columns (total 26 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 id 14606 non-null object 1 channel_sales 14606 non-null object 2 cons_12m 14606 non-null int64 3 cons_gas_12m 14606 non-null int64 4 cons_last_month 14606 non-null int64 5 date_activ 14606 non-null object 6 date_end 14606 non-null object 7 date_modif_prod 14606 non-null object 8 date_renewal 14606 non-null object 9 forecast_cons_12m 14606 non-null float64 10 forecast_cons_year 14606 non-null int64 11 forecast_discount_energy 14606 non-null float64 12 forecast_meter_rent_12m 14606 non-null float64 13 forecast_price_energy_off_peak 14606 non-null float64 14 forecast_price_energy_peak 14606 non-null float64 15 forecast_price_pow_off_peak 14606 non-null float64 16 has_gas 14606 non-null object 17 imp_cons 14606 non-null float64 18 margin_gross_pow_ele 14606 non-null float64 19 margin_net_pow_ele 14606 non-null float64 20 nb_prod_act 14606 non-null int64 21 net_margin 14606 non-null float64 22 num_years_antig 14606 non-null int64 23 origin_up 14606 non-null object 24 pow_max 14606 non-null float64 25 churn 14606 non-null int64 dtypes: float64(11), int64(7), object(8) memory usage: 2.9+ MB
dfp.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 193002 entries, 0 to 193001 Data columns (total 8 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 id 193002 non-null object 1 price_date 193002 non-null object 2 price_off_peak_var 193002 non-null float64 3 price_peak_var 193002 non-null float64 4 price_mid_peak_var 193002 non-null float64 5 price_off_peak_fix 193002 non-null float64 6 price_peak_fix 193002 non-null float64 7 price_mid_peak_fix 193002 non-null float64 dtypes: float64(6), object(2) memory usage: 11.8+ MB
we notice that we have ids of companies in the prices dataset not present in the clients dataset. we will work on this later on
msno.matrix(dfc)
<AxesSubplot:>
msno.matrix(dfp)
<AxesSubplot:>
res = dfc.isin(['MISSING']).any().any()
if res:
print("there are missing values in our dataset")
else:
print("there are no missning values in our dataset")
there are missing values in our dataset
res = dfc.isin(['MISSING']).any()
res
id False channel_sales True cons_12m False cons_gas_12m False cons_last_month False date_activ False date_end False date_modif_prod False date_renewal False forecast_cons_12m False forecast_cons_year False forecast_discount_energy False forecast_meter_rent_12m False forecast_price_energy_off_peak False forecast_price_energy_peak False forecast_price_pow_off_peak False has_gas False imp_cons False margin_gross_pow_ele False margin_net_pow_ele False nb_prod_act False net_margin False num_years_antig False origin_up True pow_max False churn False dtype: bool
dfc[dfc['channel_sales']=='MISSING']
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | d29c2c54acc38ff3c0614d0a653813dd | MISSING | 4660 | 0 | 0 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 189.95 | 0 | 0.0 | 16.27 | 0.145711 | 0.000000 | 44.311378 | f | 0.00 | 16.38 | 16.38 | 1 | 18.89 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.800 | 0 |
| 4 | 149d57cf92fc41cf94415803a877cb4b | MISSING | 4425 | 0 | 526 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 445.75 | 526 | 0.0 | 131.73 | 0.116900 | 0.100015 | 40.606701 | f | 52.32 | 44.91 | 44.91 | 1 | 47.98 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 19.800 | 0 |
| 14 | c5037389dd8be351d3c40f2973da2391 | MISSING | 9707 | 0 | 0 | 2010-12-01 | 2016-12-01 | 2010-12-01 | 2015-12-02 | 1028.52 | 0 | 0.0 | 15.37 | 0.168242 | 0.089602 | 45.308378 | f | 0.00 | 15.78 | 15.78 | 1 | 111.59 | 5 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.856 | 0 |
| 16 | bd14bbac3aca710ee5288fcd8511d4bb | MISSING | 786536 | 80583 | 85883 | 2010-02-15 | 2016-02-15 | 2010-02-15 | 2015-02-16 | 13517.50 | 23298 | 0.0 | 16.60 | 0.164058 | 0.086163 | 44.311378 | t | 2585.38 | 25.68 | 25.68 | 2 | 1087.76 | 6 | lxidpiddsbxsbosboudacockeimpuepw | 11.400 | 0 |
| 18 | f2b93783adecf0d0d8d60ab547bda3de | MISSING | 24838 | 0 | 2877 | 2011-05-27 | 2016-05-27 | 2011-05-27 | 2015-05-31 | 1088.35 | 798 | 0.0 | 141.79 | 0.115174 | 0.098837 | 40.606701 | f | 79.36 | 21.57 | 21.57 | 1 | 117.44 | 5 | ldkssxwpmemidmecebumciepifcamkci | 15.001 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14585 | cf8fdee748705aacc79b92eca825218f | MISSING | 1185 | 0 | 0 | 2009-06-08 | 2016-06-08 | 2009-06-08 | 2015-06-12 | 176.56 | 0 | 0.0 | 7.59 | 0.144902 | 0.000000 | 44.311378 | f | 0.00 | 21.52 | 21.52 | 1 | 16.13 | 7 | ldkssxwpmemidmecebumciepifcamkci | 13.200 | 0 |
| 14594 | 9c93ea61edcaddf27333bc25178e760f | MISSING | 6694 | 0 | 0 | 2011-12-01 | 2016-12-01 | 2015-11-01 | 2015-12-02 | 1007.35 | 0 | 0.0 | 15.98 | 0.144039 | 0.000000 | 45.308378 | f | 0.00 | 15.78 | 15.78 | 1 | 86.54 | 4 | ldkssxwpmemidmecebumciepifcamkci | 13.856 | 0 |
| 14595 | a6096551a64c47cd240c356b0669f892 | MISSING | 11563 | 0 | 0 | 2010-02-08 | 2016-02-08 | 2015-12-16 | 2015-02-09 | 1384.83 | 0 | 0.0 | 16.16 | 0.164641 | 0.086742 | 44.311378 | f | 0.00 | 31.20 | 31.20 | 1 | 122.35 | 6 | ldkssxwpmemidmecebumciepifcamkci | 13.856 | 0 |
| 14598 | 1582ef35fbfa265e60bb3399bdebac87 | MISSING | 9441 | 0 | 480 | 2009-10-08 | 2016-10-08 | 2015-05-24 | 2015-10-09 | 983.97 | 480 | 0.0 | 132.11 | 0.115237 | 0.100123 | 40.939027 | f | 46.98 | 20.00 | 20.00 | 1 | 113.17 | 6 | ldkssxwpmemidmecebumciepifcamkci | 15.001 | 0 |
| 14605 | 563dde550fd624d7352f3de77c0cdfcd | MISSING | 8730 | 0 | 0 | 2009-12-18 | 2016-12-17 | 2009-12-18 | 2015-12-21 | 762.41 | 0 | 0.0 | 1.07 | 0.167086 | 0.088454 | 45.311378 | f | 0.00 | 11.84 | 11.84 | 1 | 96.34 | 6 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
3725 rows × 26 columns
dfc[dfc['origin_up']=='MISSING']
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 17 | 4e4a34a6d91ff25d0fdf68733907ad44 | foosdfpfkusacimwkcsosbicdxkicaua | 30776 | 0 | 0 | 2011-08-01 | 2016-07-25 | 2014-07-25 | 2015-07-27 | 3045.16 | 0 | 0.0 | 118.24 | 0.112860 | 0.096521 | 40.606701 | f | 0.00 | 19.29 | 19.29 | 1 | 335.78 | 5 | MISSING | 25.000 | 0 |
| 158 | 01fbea88102b46f036ff17ebe4d23459 | lmkebamcaaclubfxadlmueccxoimlema | 15040 | 4383 | 0 | 2010-05-07 | 2016-04-16 | 2015-04-15 | 2015-04-17 | 1625.53 | 0 | 0.0 | 116.84 | 0.117496 | 0.101155 | 40.606701 | t | 0.00 | 30.18 | 30.18 | 2 | 204.98 | 6 | MISSING | 25.000 | 0 |
| 357 | 9c4bde81b5b4913b46c28986b39be5ae | MISSING | 30513 | 0 | 2294 | 2008-02-11 | 2016-02-11 | 2014-05-05 | 2015-02-12 | 3081.42 | 2294 | 0.0 | 131.43 | 0.116900 | 0.100015 | 40.606701 | f | 225.73 | 49.44 | 49.44 | 1 | 321.18 | 8 | MISSING | 33.000 | 0 |
| 499 | f1d5af5e1fff4c8acaac81dce31028d0 | MISSING | 17855 | 315 | 0 | 2012-04-02 | 2016-06-01 | 2015-06-01 | 2015-04-04 | 658.67 | 0 | 0.0 | 18.37 | 0.199230 | 0.000000 | 45.806878 | t | 0.00 | 37.74 | 37.74 | 3 | 240.86 | 4 | MISSING | 12.000 | 0 |
| 1094 | 66ba1d7066c1d430cc17974387c10b25 | ewpakwlliwisiwduibdlfmalxowmwpci | 2501 | 0 | 0 | 2012-12-06 | 2016-12-06 | 2015-11-01 | 2015-12-07 | 287.81 | 0 | 0.0 | 0.00 | 0.144038 | 0.000000 | 46.305378 | f | 0.00 | 27.28 | 27.28 | 1 | 21.84 | 3 | MISSING | 13.856 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 12762 | c854b26113307311c6dd2319ed1f613d | foosdfpfkusacimwkcsosbicdxkicaua | 1141 | 0 | 9 | 2012-08-30 | 2016-05-08 | 2015-11-25 | 2015-05-13 | 98.14 | 9 | 0.0 | 95.87 | 0.112860 | 0.096521 | 40.606701 | f | 0.91 | 31.41 | 31.41 | 1 | 31.93 | 3 | MISSING | 17.321 | 0 |
| 13244 | 08194f6221bfc93f6b174f4f3194db42 | MISSING | 4 | 0 | 0 | 2010-10-21 | 2016-10-21 | 2010-10-21 | 2015-10-23 | 0.60 | 0 | 0.0 | 18.62 | 0.147506 | 0.000000 | 46.305378 | f | 0.00 | 28.76 | 28.76 | 1 | 0.04 | 5 | MISSING | 13.856 | 0 |
| 13314 | 4dbafe448d59092e174999b04b97b20f | MISSING | 17493 | 0 | 1901 | 2010-11-05 | 2017-01-11 | 2016-01-11 | 2015-11-05 | 1590.92 | 1901 | 0.0 | 136.52 | 0.096347 | 0.088776 | 58.995952 | f | 159.84 | 0.00 | 0.00 | 1 | 204.66 | 5 | MISSING | 10.000 | 0 |
| 14228 | adff89602e947091aa5adc5e39f2b146 | lmkebamcaaclubfxadlmueccxoimlema | 25551 | 0 | 1891 | 2011-09-16 | 2016-09-15 | 2011-09-16 | 2015-09-16 | 20.80 | 209 | 0.0 | 188.91 | 0.115237 | 0.100123 | 41.271364 | f | 19.89 | 72.15 | 72.15 | 1 | 0.11 | 4 | MISSING | 35.000 | 0 |
| 14352 | ede19a19e735f1f24416966c93e5b251 | foosdfpfkusacimwkcsosbicdxkicaua | 70948 | 0 | 0 | 2013-01-08 | 2016-05-08 | 2014-04-28 | 2015-05-11 | 1840.29 | 0 | 0.0 | 13.29 | 0.161454 | 0.084198 | 44.311378 | f | 0.00 | 17.22 | 17.22 | 1 | 165.07 | 3 | MISSING | 10.392 | 0 |
64 rows × 26 columns
Our main task here is to understand the influence of the prices on the customers churn. So to understand the correlations between these factors we need to join both of the dataframes in a way that we have the id of the customer in a row followed by their info and then their prices info
Let's start visualizing our data and we will begin by determining the correlations between our features to further analyze them
churn = dfc[['id', 'churn']]
churn.columns = ['Companies', 'churn']
churn_total = churn.groupby(churn['churn']).count()
churn_percentage = churn_total / churn_total.sum() * 100
fig = px.bar(churn_percentage.transpose() ,barmode='stack', title="Churn Distribution",text_auto=True)
fig.show()
df_corr = dfc.corr()
matrix = np.triu(df_corr)
fig, ax = plt.subplots(figsize=(23,10))
sns.heatmap(df_corr, vmin=-1, vmax=1, annot=True,ax=ax)
<AxesSubplot:>
channel = dfc[['id', 'channel_sales', 'churn']]
channel = channel.groupby([channel['channel_sales'], channel['churn']])['id'].count().unstack(level=1).fillna(0)
channel_churn = (channel.div(channel.sum(axis=1), axis=0) * 100).sort_values(by=[1], ascending=False)
fig = px.bar(channel_churn,barmode='stack',text_auto=True)
fig.show()
consumption = dfc[['id', 'cons_12m', 'cons_gas_12m', 'cons_last_month', 'imp_cons', 'has_gas', 'churn']]
fig = px.histogram(consumption,x="cons_12m",color='churn',barmode='stack',pattern_shape="churn")
fig.show()
fig = px.histogram(consumption,x="has_gas",color='churn',barmode='stack',pattern_shape="churn")
fig.show()
Our dataset is highly imbalanced as we have nearly 1/8 of our customers churning and 7/8 who are not overall.
fig = px.histogram(consumption,x="cons_gas_12m",color='churn',barmode='stack',pattern_shape="churn")
fig.show()
fig = px.histogram(consumption,x="cons_last_month",color='churn',barmode='stack',pattern_shape="churn")
fig.show()
fig = px.histogram(consumption,x="imp_cons",color='churn',barmode='stack',pattern_shape="churn")
fig.show()
fig=px.scatter(consumption,x='imp_cons',color="churn")
fig.show()
fig=px.scatter(consumption,x='has_gas',color="churn")
fig.show()
fig=px.scatter(consumption,x='cons_12m',color="churn")
fig.show()
fig=px.scatter(consumption,x='cons_last_month',color="churn")
fig.show()
Through these charts we can clearly notice 2 major things :
px.scatter(data_frame=dfc,x="cons_12m", y="forecast_cons_12m",
color = 'churn')
This chart showcases the inaccuracy of the forcasted consumption in the past 12 months compared to the actual consumption in the last 12 months
px.scatter(data_frame=dfc,x="imp_cons", y="forecast_cons_year",
color = 'churn')
we can clearly see here the linearity in the paid consumption and the forcasted consumption in a year
fig.update_layout(template='seaborn')
px.scatter(data_frame=dfc,x="num_years_antig", y="cons_12m",
color = 'churn')
The antiquity of the customers reflect their consumption and as we can see the density of the consumption is overall located in those who have been customers for 3-6 years
margin = dfc[['id', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'net_margin']]
fig = px.box(data_frame=dfc,x='margin_gross_pow_ele')
fig.show()
fig = px.box(data_frame=dfc,x='margin_net_pow_ele')
fig.show()
fig = px.box(data_frame=dfc,x='net_margin')
fig.show()
similarly to the margin we detect the presence of outliers that will be handled
others = dfc[['id', 'nb_prod_act', 'num_years_antig', 'origin_up', 'churn']]
products = others.groupby([others["nb_prod_act"],others["churn"]])["id"].count().unstack(level=1)
products_percentage = (products.div(products.sum(axis=1), axis=0)*100).sort_values(by=[1], ascending=False)
fig = px.bar(products_percentage.transpose() ,barmode='stack', title="Churn number of active products",text_auto=True)
fig.show()
years_antig = others.groupby([others["num_years_antig"],others["churn"]])["id"].count().unstack(level=1)
years_antig_percentage = (years_antig.div(years_antig.sum(axis=1), axis=0)*100)
fig = px.bar(years_antig_percentage.transpose() ,barmode='stack', title="Churn number of active products",text_auto=True)
fig.show()
# let's convert the date columns to valid date columns
dfc["date_activ"] = pd.to_datetime(dfc["date_activ"], format='%Y-%m-%d')
dfc["date_end"] = pd.to_datetime(dfc["date_end"], format='%Y-%m-%d')
dfc["date_modif_prod"] = pd.to_datetime(dfc["date_modif_prod"], format='%Y-%m-%d')
dfc["date_renewal"] = pd.to_datetime(dfc["date_renewal"], format='%Y-%m-%d')
dfp['price_date'] = pd.to_datetime(dfp['price_date'], format='%Y-%m-%d')
# Now let us create the mean data frames
mean_year = dfp.groupby(['id']).mean().reset_index()
mean_6m = dfp[dfp['price_date'] > '2015-06-01'].groupby(['id']).mean().reset_index()
mean_3m = dfp[dfp['price_date'] > '2015-10-01'].groupby(['id']).mean().reset_index()
# moving onto combining into single dataframe
mean_year = mean_year.rename(
index=str,
columns={
"price_off_peak_var": "mean_year_price_p1_var",
"price_peak_var": "mean_year_price_p2_var",
"price_mid_peak_var": "mean_year_price_p3_var",
"price_off_peak_fix": "mean_year_price_p1_fix",
"price_peak_fix": "mean_year_price_p2_fix",
"price_mid_peak_fix": "mean_year_price_p3_fix"
}
)
mean_year["mean_year_price_p1"] = mean_year["mean_year_price_p1_var"] + mean_year["mean_year_price_p1_fix"]
mean_year["mean_year_price_p2"] = mean_year["mean_year_price_p2_var"] + mean_year["mean_year_price_p2_fix"]
mean_year["mean_year_price_p3"] = mean_year["mean_year_price_p3_var"] + mean_year["mean_year_price_p3_fix"]
mean_6m = mean_6m.rename(
index=str,
columns={
"price_off_peak_var": "mean_6m_price_p1_var",
"price_peak_var": "mean_6m_price_p2_var",
"price_mid_peak_var": "mean_6m_price_p3_var",
"price_off_peak_fix": "mean_6m_price_p1_fix",
"price_peak_fix": "mean_6m_price_p2_fix",
"price_mid_peak_fix": "mean_6m_price_p3_fix"
}
)
mean_6m["mean_6m_price_p1"] = mean_6m["mean_6m_price_p1_var"] + mean_6m["mean_6m_price_p1_fix"]
mean_6m["mean_6m_price_p2"] = mean_6m["mean_6m_price_p2_var"] + mean_6m["mean_6m_price_p2_fix"]
mean_6m["mean_6m_price_p3"] = mean_6m["mean_6m_price_p3_var"] + mean_6m["mean_6m_price_p3_fix"]
mean_3m = mean_3m.rename(
index=str,
columns={
"price_off_peak_var": "mean_3m_price_p1_var",
"price_peak_var": "mean_3m_price_p2_var",
"price_mid_peak_var": "mean_3m_price_p3_var",
"price_off_peak_fix": "mean_3m_price_p1_fix",
"price_peak_fix": "mean_3m_price_p2_fix",
"price_mid_peak_fix": "mean_3m_price_p3_fix"
}
)
mean_3m["mean_3m_price_p1"] = mean_3m["mean_3m_price_p1_var"] + mean_3m["mean_3m_price_p1_fix"]
mean_3m["mean_3m_price_p2"] = mean_3m["mean_3m_price_p2_var"] + mean_3m["mean_3m_price_p2_fix"]
mean_3m["mean_3m_price_p3"] = mean_3m["mean_3m_price_p3_var"] + mean_3m["mean_3m_price_p3_fix"]
# Now let's merge them into a final dataframe
price_features = pd.merge(mean_year, mean_6m, on='id')
price_features = pd.merge(price_features, mean_3m, on='id')
price_features
| id | mean_year_price_p1_var | mean_year_price_p2_var | mean_year_price_p3_var | mean_year_price_p1_fix | mean_year_price_p2_fix | mean_year_price_p3_fix | mean_year_price_p1 | mean_year_price_p2 | mean_year_price_p3 | mean_6m_price_p1_var | mean_6m_price_p2_var | mean_6m_price_p3_var | mean_6m_price_p1_fix | mean_6m_price_p2_fix | mean_6m_price_p3_fix | mean_6m_price_p1 | mean_6m_price_p2 | mean_6m_price_p3 | mean_3m_price_p1_var | mean_3m_price_p2_var | mean_3m_price_p3_var | mean_3m_price_p1_fix | mean_3m_price_p2_fix | mean_3m_price_p3_fix | mean_3m_price_p1 | mean_3m_price_p2 | mean_3m_price_p3 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0002203ffbb812588b632b9e628cc38d | 0.124338 | 0.103794 | 0.073160 | 40.701732 | 24.421038 | 16.280694 | 40.826071 | 24.524832 | 16.353854 | 0.121266 | 0.102368 | 0.073728 | 40.728885 | 24.437330 | 16.291555 | 40.850151 | 24.539698 | 16.365283 | 0.119906 | 0.101673 | 0.073719 | 40.728885 | 24.43733 | 16.291555 | 40.848791 | 24.539003 | 16.365274 |
| 1 | 0004351ebdd665e6ee664792efc4fd13 | 0.146426 | 0.000000 | 0.000000 | 44.385450 | 0.000000 | 0.000000 | 44.531877 | 0.000000 | 0.000000 | 0.144687 | 0.000000 | 0.000000 | 44.444710 | 0.000000 | 0.000000 | 44.589397 | 0.000000 | 0.000000 | 0.143943 | 0.000000 | 0.000000 | 44.444710 | 0.00000 | 0.000000 | 44.588653 | 0.000000 | 0.000000 |
| 2 | 0010bcc39e42b3c2131ed2ce55246e3c | 0.181558 | 0.000000 | 0.000000 | 45.319710 | 0.000000 | 0.000000 | 45.501268 | 0.000000 | 0.000000 | 0.202024 | 0.000000 | 0.000000 | 45.944710 | 0.000000 | 0.000000 | 46.146734 | 0.000000 | 0.000000 | 0.201280 | 0.000000 | 0.000000 | 45.944710 | 0.00000 | 0.000000 | 46.145990 | 0.000000 | 0.000000 |
| 3 | 0010ee3855fdea87602a5b7aba8e42de | 0.118757 | 0.098292 | 0.069032 | 40.647427 | 24.388455 | 16.258971 | 40.766185 | 24.486748 | 16.328003 | 0.114428 | 0.096080 | 0.069418 | 40.728885 | 24.437330 | 16.291555 | 40.843313 | 24.533410 | 16.360973 | 0.113068 | 0.095385 | 0.069409 | 40.728885 | 24.43733 | 16.291555 | 40.841953 | 24.532715 | 16.360964 |
| 4 | 00114d74e963e47177db89bc70108537 | 0.147926 | 0.000000 | 0.000000 | 44.266930 | 0.000000 | 0.000000 | 44.414856 | 0.000000 | 0.000000 | 0.146184 | 0.000000 | 0.000000 | 44.266930 | 0.000000 | 0.000000 | 44.413114 | 0.000000 | 0.000000 | 0.145440 | 0.000000 | 0.000000 | 44.266930 | 0.00000 | 0.000000 | 44.412370 | 0.000000 | 0.000000 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 16090 | ffef185810e44254c3a4c6395e6b4d8a | 0.138863 | 0.115125 | 0.080780 | 40.896427 | 24.637456 | 16.507972 | 41.035291 | 24.752581 | 16.588752 | 0.113848 | 0.095499 | 0.068838 | 40.728885 | 24.437330 | 16.291555 | 40.842733 | 24.532829 | 16.360393 | 0.112488 | 0.094804 | 0.068829 | 40.728885 | 24.43733 | 16.291555 | 40.841373 | 24.532134 | 16.360384 |
| 16091 | fffac626da707b1b5ab11e8431a4d0a2 | 0.147137 | 0.000000 | 0.000000 | 44.311375 | 0.000000 | 0.000000 | 44.458512 | 0.000000 | 0.000000 | 0.145449 | 0.000000 | 0.000000 | 44.355820 | 0.000000 | 0.000000 | 44.501269 | 0.000000 | 0.000000 | 0.145047 | 0.000000 | 0.000000 | 44.444710 | 0.00000 | 0.000000 | 44.589757 | 0.000000 | 0.000000 |
| 16092 | fffc0cacd305dd51f316424bbb08d1bd | 0.153879 | 0.129497 | 0.094842 | 41.160171 | 24.895768 | 16.763569 | 41.314049 | 25.025265 | 16.858411 | 0.152759 | 0.127566 | 0.091403 | 41.228885 | 24.937330 | 16.791555 | 41.381645 | 25.064896 | 16.882958 | 0.151399 | 0.126871 | 0.091394 | 41.228885 | 24.93733 | 16.791555 | 41.380284 | 25.064201 | 16.882949 |
| 16093 | fffe4f5646aa39c7f97f95ae2679ce64 | 0.123858 | 0.103499 | 0.073735 | 40.606699 | 24.364017 | 16.242678 | 40.730558 | 24.467516 | 16.316414 | 0.120150 | 0.101570 | 0.073983 | 40.647429 | 24.388454 | 16.258969 | 40.767579 | 24.490024 | 16.332952 | 0.118175 | 0.100491 | 0.074516 | 40.728885 | 24.43733 | 16.291555 | 40.847060 | 24.537821 | 16.366071 |
| 16094 | ffff7fa066f1fb305ae285bb03bf325a | 0.125360 | 0.104895 | 0.075635 | 40.647427 | 24.388455 | 16.258971 | 40.772788 | 24.493350 | 16.334606 | 0.121276 | 0.102927 | 0.076266 | 40.728885 | 24.437330 | 16.291555 | 40.850161 | 24.540257 | 16.367821 | 0.119916 | 0.102232 | 0.076257 | 40.728885 | 24.43733 | 16.291555 | 40.848801 | 24.539562 | 16.367812 |
16095 rows × 28 columns
# adding the churn column and merging with the ID
price_analysis = pd.merge(price_features, dfc[['id', 'churn']], on='id')
price_analysis.head()
| id | mean_year_price_p1_var | mean_year_price_p2_var | mean_year_price_p3_var | mean_year_price_p1_fix | mean_year_price_p2_fix | mean_year_price_p3_fix | mean_year_price_p1 | mean_year_price_p2 | mean_year_price_p3 | mean_6m_price_p1_var | mean_6m_price_p2_var | mean_6m_price_p3_var | mean_6m_price_p1_fix | mean_6m_price_p2_fix | mean_6m_price_p3_fix | mean_6m_price_p1 | mean_6m_price_p2 | mean_6m_price_p3 | mean_3m_price_p1_var | mean_3m_price_p2_var | mean_3m_price_p3_var | mean_3m_price_p1_fix | mean_3m_price_p2_fix | mean_3m_price_p3_fix | mean_3m_price_p1 | mean_3m_price_p2 | mean_3m_price_p3 | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0002203ffbb812588b632b9e628cc38d | 0.124338 | 0.103794 | 0.073160 | 40.701732 | 24.421038 | 16.280694 | 40.826071 | 24.524832 | 16.353854 | 0.121266 | 0.102368 | 0.073728 | 40.728885 | 24.43733 | 16.291555 | 40.850151 | 24.539698 | 16.365283 | 0.119906 | 0.101673 | 0.073719 | 40.728885 | 24.43733 | 16.291555 | 40.848791 | 24.539003 | 16.365274 | 0 |
| 1 | 0004351ebdd665e6ee664792efc4fd13 | 0.146426 | 0.000000 | 0.000000 | 44.385450 | 0.000000 | 0.000000 | 44.531877 | 0.000000 | 0.000000 | 0.144687 | 0.000000 | 0.000000 | 44.444710 | 0.00000 | 0.000000 | 44.589397 | 0.000000 | 0.000000 | 0.143943 | 0.000000 | 0.000000 | 44.444710 | 0.00000 | 0.000000 | 44.588653 | 0.000000 | 0.000000 | 0 |
| 2 | 0010bcc39e42b3c2131ed2ce55246e3c | 0.181558 | 0.000000 | 0.000000 | 45.319710 | 0.000000 | 0.000000 | 45.501268 | 0.000000 | 0.000000 | 0.202024 | 0.000000 | 0.000000 | 45.944710 | 0.00000 | 0.000000 | 46.146734 | 0.000000 | 0.000000 | 0.201280 | 0.000000 | 0.000000 | 45.944710 | 0.00000 | 0.000000 | 46.145990 | 0.000000 | 0.000000 | 0 |
| 3 | 00114d74e963e47177db89bc70108537 | 0.147926 | 0.000000 | 0.000000 | 44.266930 | 0.000000 | 0.000000 | 44.414856 | 0.000000 | 0.000000 | 0.146184 | 0.000000 | 0.000000 | 44.266930 | 0.00000 | 0.000000 | 44.413114 | 0.000000 | 0.000000 | 0.145440 | 0.000000 | 0.000000 | 44.266930 | 0.00000 | 0.000000 | 44.412370 | 0.000000 | 0.000000 | 0 |
| 4 | 0013f326a839a2f6ad87a1859952d227 | 0.126076 | 0.105542 | 0.074921 | 40.728885 | 24.437330 | 16.291555 | 40.854961 | 24.542872 | 16.366476 | 0.123007 | 0.104108 | 0.075469 | 40.728885 | 24.43733 | 16.291555 | 40.851892 | 24.541438 | 16.367024 | 0.121647 | 0.103413 | 0.075460 | 40.728885 | 24.43733 | 16.291555 | 40.850532 | 24.540743 | 16.367015 | 0 |
fig = px.scatter(price_analysis, x="mean_year_price_p1", y="churn", marginal_x="box", marginal_y="violin")
fig.show()
fig = px.scatter(price_analysis, x="mean_6m_price_p2", y="churn", marginal_x="box", marginal_y="violin")
fig.show()
fig = px.scatter(price_analysis, x="mean_3m_price_p3", y="churn", marginal_x="box", marginal_y="violin")
fig.show()
There is a very low positive correlation with the churn shown in the previous 3 jointgridss
corr = price_analysis.corr()
# Plot correlation
plt.figure(figsize=(20,18))
sns.heatmap(corr, xticklabels=corr.columns.values, yticklabels=corr.columns.values, annot = True, annot_kws={'size':10})
# Axis ticks size
plt.xticks(fontsize=10)
plt.yticks(fontsize=10)
plt.show()
As shown in the exploratory data analysis phase our numerical features are highly skewed . We need to handle this skewness since most of the predictive models typically assume that the data is independent and normally distributed.
Some of the techniques to use for skewness handling are Square root - Cubic root - Logarithm. in our case we will use the logarithm technique for positive high skewness
#extracting the skewed columns
skewed_columns = [
'cons_12m',
'cons_gas_12m',
'cons_last_month',
'forecast_cons_12m',
'forecast_cons_year',
'forecast_discount_energy',
'forecast_meter_rent_12m',
'forecast_price_energy_off_peak',
'forecast_price_energy_peak',
'forecast_price_pow_off_peak'
]
# Apply inglog10 transformation
dfc["cons_12m"] = np.sqrt(dfc["cons_12m"] )
dfc["cons_gas_12m"] = np.sqrt(dfc["cons_gas_12m"] )
dfc["cons_last_month"] = np.sqrt(dfc["cons_last_month"] )
dfc["forecast_cons_12m"] = np.sqrt(dfc["forecast_cons_12m"] )
dfc["forecast_cons_year"] = np.sqrt(dfc["forecast_cons_year"] )
dfc["forecast_meter_rent_12m"] = np.sqrt(dfc["forecast_meter_rent_12m"])
dfc["imp_cons"] = np.sqrt(dfc["imp_cons"] )
Now let's see some of the new distributions
dfc
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 24011ae4ebbe3035111d65fa7c15bc57 | foosdfpfkusacimwkcsosbicdxkicaua | 0.000000 | 234.405631 | 0.000000 | 2013-06-15 | 2016-06-15 | 2015-11-01 | 2015-06-23 | 0.000000 | 0.000000 | 0.0 | 1.334166 | 0.114481 | 0.098142 | 40.606701 | t | 0.000000 | 25.44 | 25.44 | 2 | 678.99 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 43.648 | 1 |
| 1 | d29c2c54acc38ff3c0614d0a653813dd | MISSING | 68.264193 | 0.000000 | 0.000000 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 13.782235 | 0.000000 | 0.0 | 4.033609 | 0.145711 | 0.000000 | 44.311378 | f | 0.000000 | 16.38 | 16.38 | 1 | 18.89 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.800 | 0 |
| 2 | 764c75f661154dac3a6c254cd082ea7d | foosdfpfkusacimwkcsosbicdxkicaua | 23.323808 | 0.000000 | 0.000000 | 2010-04-16 | 2016-04-16 | 2010-04-16 | 2015-04-17 | 6.925316 | 0.000000 | 0.0 | 6.222540 | 0.165794 | 0.087899 | 44.311378 | f | 0.000000 | 28.60 | 28.60 | 1 | 6.60 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.856 | 0 |
| 3 | bba03439a292a1e166f80264c16191cb | lmkebamcaaclubfxadlmueccxoimlema | 39.799497 | 0.000000 | 0.000000 | 2010-03-30 | 2016-03-30 | 2010-03-30 | 2015-03-31 | 15.493224 | 0.000000 | 0.0 | 4.453089 | 0.146694 | 0.000000 | 44.311378 | f | 0.000000 | 30.22 | 30.22 | 1 | 25.46 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 4 | 149d57cf92fc41cf94415803a877cb4b | MISSING | 66.520673 | 0.000000 | 22.934690 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 21.112792 | 22.934690 | 0.0 | 11.477369 | 0.116900 | 0.100015 | 40.606701 | f | 7.233257 | 44.91 | 44.91 | 1 | 47.98 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 19.800 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14601 | 18463073fb097fc0ac5d3e040f356987 | foosdfpfkusacimwkcsosbicdxkicaua | 179.638526 | 218.952050 | 0.000000 | 2012-05-24 | 2016-05-08 | 2015-05-08 | 2014-05-26 | 68.176316 | 0.000000 | 0.0 | 4.309292 | 0.138305 | 0.000000 | 44.311378 | t | 0.000000 | 27.88 | 27.88 | 2 | 381.77 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 15.000 | 0 |
| 14602 | d0a6f71671571ed83b2645d23af6de00 | foosdfpfkusacimwkcsosbicdxkicaua | 84.988234 | 0.000000 | 13.453624 | 2012-08-27 | 2016-08-27 | 2012-08-27 | 2015-08-28 | 25.133444 | 13.453624 | 0.0 | 12.001250 | 0.100167 | 0.091892 | 58.995952 | f | 3.992493 | 0.00 | 0.00 | 1 | 90.34 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 6.000 | 1 |
| 14603 | 10e6828ddd62cbcf687cb74928c4c2d2 | foosdfpfkusacimwkcsosbicdxkicaua | 42.941821 | 0.000000 | 13.379088 | 2012-02-08 | 2016-02-07 | 2012-02-08 | 2015-02-09 | 13.798188 | 13.379088 | 0.0 | 11.384200 | 0.116900 | 0.100015 | 40.606701 | f | 4.248529 | 39.84 | 39.84 | 1 | 20.38 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 15.935 | 1 |
| 14604 | 1cf20fd6206d7678d5bcafd28c53b4db | foosdfpfkusacimwkcsosbicdxkicaua | 11.445523 | 0.000000 | 0.000000 | 2012-08-30 | 2016-08-30 | 2012-08-30 | 2015-08-31 | 4.397727 | 0.000000 | 0.0 | 2.679552 | 0.145711 | 0.000000 | 44.311378 | f | 0.000000 | 13.08 | 13.08 | 1 | 0.96 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 11.000 | 0 |
| 14605 | 563dde550fd624d7352f3de77c0cdfcd | MISSING | 93.434469 | 0.000000 | 0.000000 | 2009-12-18 | 2016-12-17 | 2009-12-18 | 2015-12-21 | 27.611773 | 0.000000 | 0.0 | 1.034408 | 0.167086 | 0.088454 | 45.311378 | f | 0.000000 | 11.84 | 11.84 | 1 | 96.34 | 6 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
14606 rows × 26 columns
fig, axs = plt.subplots(nrows=3, figsize=(18, 20))
sns.distplot((dfc["cons_12m"].dropna()), ax=axs[0])
sns.distplot((dfc[dfc["has_gas"]=='t']["cons_gas_12m"].dropna()), ax=axs[1])
sns.distplot((dfc["cons_last_month"].dropna()), ax=axs[2])
plt.show()
This technique handled the skewness quite well ! Now we have a normal distribution almost on most of our features
dfc = dfc[dfc["forecast_cons_12m"]<40000]
dfc = dfc[dfc["imp_cons"]<9000]
dfc = dfc[dfc["margin_gross_pow_ele"]<230]
dfc = dfc[dfc["margin_net_pow_ele"]<230]
dfc = dfc[dfc["net_margin"]<5000]
dfc.shape
(14600, 26)
dfc.reset_index(drop=True,inplace=True)
channel_sales_freq = dfc['channel_sales'].value_counts()
channel_sales_freq.mode()
0 2 1 3 2 11 3 893 4 1374 5 1843 6 3724 7 6750 Name: channel_sales, dtype: int64
# we will fill the missing value with the most frequent channel_sales
dfc.loc[dfc["channel_sales"]=="MISSING","channel_sales"]="foosdfpfkusacimwkcsosbicdxkicaua"
dfc["channel_sales"].unique()
array(['foosdfpfkusacimwkcsosbicdxkicaua',
'lmkebamcaaclubfxadlmueccxoimlema',
'usilxuppasemubllopkaafesmlibmsdf',
'ewpakwlliwisiwduibdlfmalxowmwpci',
'epumfxlbckeskwekxbiuasklxalciiuu',
'sddiedcslfslkckwlfkdpoeeailfpeds',
'fixdbufsefwooaasfcxdxadsiekoceaa'], dtype=object)
We will start by converting the IDs into numbers
dfc
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 24011ae4ebbe3035111d65fa7c15bc57 | foosdfpfkusacimwkcsosbicdxkicaua | 0.000000 | 234.405631 | 0.000000 | 2013-06-15 | 2016-06-15 | 2015-11-01 | 2015-06-23 | 0.000000 | 0.000000 | 0.0 | 1.334166 | 0.114481 | 0.098142 | 40.606701 | t | 0.000000 | 25.44 | 25.44 | 2 | 678.99 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 43.648 | 1 |
| 1 | d29c2c54acc38ff3c0614d0a653813dd | foosdfpfkusacimwkcsosbicdxkicaua | 68.264193 | 0.000000 | 0.000000 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 13.782235 | 0.000000 | 0.0 | 4.033609 | 0.145711 | 0.000000 | 44.311378 | f | 0.000000 | 16.38 | 16.38 | 1 | 18.89 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.800 | 0 |
| 2 | 764c75f661154dac3a6c254cd082ea7d | foosdfpfkusacimwkcsosbicdxkicaua | 23.323808 | 0.000000 | 0.000000 | 2010-04-16 | 2016-04-16 | 2010-04-16 | 2015-04-17 | 6.925316 | 0.000000 | 0.0 | 6.222540 | 0.165794 | 0.087899 | 44.311378 | f | 0.000000 | 28.60 | 28.60 | 1 | 6.60 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.856 | 0 |
| 3 | bba03439a292a1e166f80264c16191cb | lmkebamcaaclubfxadlmueccxoimlema | 39.799497 | 0.000000 | 0.000000 | 2010-03-30 | 2016-03-30 | 2010-03-30 | 2015-03-31 | 15.493224 | 0.000000 | 0.0 | 4.453089 | 0.146694 | 0.000000 | 44.311378 | f | 0.000000 | 30.22 | 30.22 | 1 | 25.46 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 13.200 | 0 |
| 4 | 149d57cf92fc41cf94415803a877cb4b | foosdfpfkusacimwkcsosbicdxkicaua | 66.520673 | 0.000000 | 22.934690 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 21.112792 | 22.934690 | 0.0 | 11.477369 | 0.116900 | 0.100015 | 40.606701 | f | 7.233257 | 44.91 | 44.91 | 1 | 47.98 | 6 | kamkkxfxxuwbdslkwifmmcsiusiuosws | 19.800 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14595 | 18463073fb097fc0ac5d3e040f356987 | foosdfpfkusacimwkcsosbicdxkicaua | 179.638526 | 218.952050 | 0.000000 | 2012-05-24 | 2016-05-08 | 2015-05-08 | 2014-05-26 | 68.176316 | 0.000000 | 0.0 | 4.309292 | 0.138305 | 0.000000 | 44.311378 | t | 0.000000 | 27.88 | 27.88 | 2 | 381.77 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 15.000 | 0 |
| 14596 | d0a6f71671571ed83b2645d23af6de00 | foosdfpfkusacimwkcsosbicdxkicaua | 84.988234 | 0.000000 | 13.453624 | 2012-08-27 | 2016-08-27 | 2012-08-27 | 2015-08-28 | 25.133444 | 13.453624 | 0.0 | 12.001250 | 0.100167 | 0.091892 | 58.995952 | f | 3.992493 | 0.00 | 0.00 | 1 | 90.34 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 6.000 | 1 |
| 14597 | 10e6828ddd62cbcf687cb74928c4c2d2 | foosdfpfkusacimwkcsosbicdxkicaua | 42.941821 | 0.000000 | 13.379088 | 2012-02-08 | 2016-02-07 | 2012-02-08 | 2015-02-09 | 13.798188 | 13.379088 | 0.0 | 11.384200 | 0.116900 | 0.100015 | 40.606701 | f | 4.248529 | 39.84 | 39.84 | 1 | 20.38 | 4 | lxidpiddsbxsbosboudacockeimpuepw | 15.935 | 1 |
| 14598 | 1cf20fd6206d7678d5bcafd28c53b4db | foosdfpfkusacimwkcsosbicdxkicaua | 11.445523 | 0.000000 | 0.000000 | 2012-08-30 | 2016-08-30 | 2012-08-30 | 2015-08-31 | 4.397727 | 0.000000 | 0.0 | 2.679552 | 0.145711 | 0.000000 | 44.311378 | f | 0.000000 | 13.08 | 13.08 | 1 | 0.96 | 3 | lxidpiddsbxsbosboudacockeimpuepw | 11.000 | 0 |
| 14599 | 563dde550fd624d7352f3de77c0cdfcd | foosdfpfkusacimwkcsosbicdxkicaua | 93.434469 | 0.000000 | 0.000000 | 2009-12-18 | 2016-12-17 | 2009-12-18 | 2015-12-21 | 27.611773 | 0.000000 | 0.0 | 1.034408 | 0.167086 | 0.088454 | 45.311378 | f | 0.000000 | 11.84 | 11.84 | 1 | 96.34 | 6 | ldkssxwpmemidmecebumciepifcamkci | 10.392 | 0 |
14600 rows × 26 columns
We will now be using target encoding for the channel sales feature using ordinal encoding
from sklearn.preprocessing import OrdinalEncoder
ord_enc = OrdinalEncoder()
dfc["channel_sales"] = ord_enc.fit_transform(dfc[["channel_sales"]])
ord_enc = OrdinalEncoder()
dfc["id"] = ord_enc.fit_transform(dfc[["id"]])
dfc["origin_up"] = ord_enc.fit_transform(dfc[["origin_up"]])
dfc["has_gas"] = ord_enc.fit_transform(dfc[["has_gas"]])
dfc
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2116.0 | 3.0 | 0.000000 | 234.405631 | 0.000000 | 2013-06-15 | 2016-06-15 | 2015-11-01 | 2015-06-23 | 0.000000 | 0.000000 | 0.0 | 1.334166 | 0.114481 | 0.098142 | 40.606701 | 1.0 | 0.000000 | 25.44 | 25.44 | 2 | 678.99 | 3 | 4.0 | 43.648 | 1 |
| 1 | 12004.0 | 3.0 | 68.264193 | 0.000000 | 0.000000 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 13.782235 | 0.000000 | 0.0 | 4.033609 | 0.145711 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 16.38 | 16.38 | 1 | 18.89 | 6 | 2.0 | 13.800 | 0 |
| 2 | 6713.0 | 3.0 | 23.323808 | 0.000000 | 0.000000 | 2010-04-16 | 2016-04-16 | 2010-04-16 | 2015-04-17 | 6.925316 | 0.000000 | 0.0 | 6.222540 | 0.165794 | 0.087899 | 44.311378 | 0.0 | 0.000000 | 28.60 | 28.60 | 1 | 6.60 | 6 | 2.0 | 13.856 | 0 |
| 3 | 10657.0 | 4.0 | 39.799497 | 0.000000 | 0.000000 | 2010-03-30 | 2016-03-30 | 2010-03-30 | 2015-03-31 | 15.493224 | 0.000000 | 0.0 | 4.453089 | 0.146694 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 30.22 | 30.22 | 1 | 25.46 | 6 | 2.0 | 13.200 | 0 |
| 4 | 1194.0 | 3.0 | 66.520673 | 0.000000 | 22.934690 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 21.112792 | 22.934690 | 0.0 | 11.477369 | 0.116900 | 0.100015 | 40.606701 | 0.0 | 7.233257 | 44.91 | 44.91 | 1 | 47.98 | 6 | 2.0 | 19.800 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14595 | 1409.0 | 3.0 | 179.638526 | 218.952050 | 0.000000 | 2012-05-24 | 2016-05-08 | 2015-05-08 | 2014-05-26 | 68.176316 | 0.000000 | 0.0 | 4.309292 | 0.138305 | 0.000000 | 44.311378 | 1.0 | 0.000000 | 27.88 | 27.88 | 2 | 381.77 | 4 | 4.0 | 15.000 | 0 |
| 14596 | 11880.0 | 3.0 | 84.988234 | 0.000000 | 13.453624 | 2012-08-27 | 2016-08-27 | 2012-08-27 | 2015-08-28 | 25.133444 | 13.453624 | 0.0 | 12.001250 | 0.100167 | 0.091892 | 58.995952 | 0.0 | 3.992493 | 0.00 | 0.00 | 1 | 90.34 | 3 | 4.0 | 6.000 | 1 |
| 14597 | 996.0 | 3.0 | 42.941821 | 0.000000 | 13.379088 | 2012-02-08 | 2016-02-07 | 2012-02-08 | 2015-02-09 | 13.798188 | 13.379088 | 0.0 | 11.384200 | 0.116900 | 0.100015 | 40.606701 | 0.0 | 4.248529 | 39.84 | 39.84 | 1 | 20.38 | 4 | 4.0 | 15.935 | 1 |
| 14598 | 1676.0 | 3.0 | 11.445523 | 0.000000 | 0.000000 | 2012-08-30 | 2016-08-30 | 2012-08-30 | 2015-08-31 | 4.397727 | 0.000000 | 0.0 | 2.679552 | 0.145711 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 13.08 | 13.08 | 1 | 0.96 | 3 | 4.0 | 11.000 | 0 |
| 14599 | 4939.0 | 3.0 | 93.434469 | 0.000000 | 0.000000 | 2009-12-18 | 2016-12-17 | 2009-12-18 | 2015-12-21 | 27.611773 | 0.000000 | 0.0 | 1.034408 | 0.167086 | 0.088454 | 45.311378 | 0.0 | 0.000000 | 11.84 | 11.84 | 1 | 96.34 | 6 | 3.0 | 10.392 | 0 |
14600 rows × 26 columns
dfc.drop([14596,14597,14598,14599],inplace=True)
Now that we have all of our data in numerical format let's conduct a further outliers handling using the Isolation Forest algorithm.
The Isolation Forest algorithm is a fast tree-based algorithm for anomaly detection. it takes advantage of the following properties :
Fewness : anomalous samples are a minority and there will only be a few of them in any dataset
Different : anomalous samples have values/attributes that are very different from those of normal samples
from sklearn.ensemble import IsolationForest
# def remove_outliers(df):
# clf = IsolationForest(random_state=0).fit(df)
# anomaly = clf.predict(df)
# df['anomaly']= anomaly
# outliers=df.loc[df['anomaly']==-1]
# outlier_index=list(outliers.index)
# print(df['anomaly'].value_counts())
# df = df[df['anomaly'] != -1]
# return df.loc[df['anomaly']== 1]
# dfc = remove_outliers(dfc)
# dfc.drop(['anomaly'], axis=1, inplace=True)
We will follow a grouping approach for feature engineering and we will be creating the following features
dfc['total_cons_12m'] = dfc['cons_12m'] + dfc['cons_gas_12m']
dfc['fore_casted_price_energy_total'] = dfc['forecast_price_energy_off_peak'] + dfc['forecast_price_energy_peak']
handling date features
def extract_date(df,date_col_name):
#extracting year, month, day
for i in range(len(df)):
dates = df[date_col_name]
date = dates[i].to_pydatetime().date()
year = date.year
month = date.month
day = date.day
#affecting the values to columns
df.loc[i,'year_'+ date_col_name] = year
df.loc[i,'month_'+ date_col_name] = month
df.loc[i,'day_'+ date_col_name] = day
return df
dfc = extract_date(dfc,"date_activ")
dfc = extract_date(dfc,"date_end")
dfc = extract_date(dfc,"date_modif_prod")
dfc = extract_date(dfc,"date_renewal")
dfc
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | date_activ | date_end | date_modif_prod | date_renewal | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | total_cons_12m | fore_casted_price_energy_total | year_date_activ | month_date_activ | day_date_activ | year_date_end | month_date_end | day_date_end | year_date_modif_prod | month_date_modif_prod | day_date_modif_prod | year_date_renewal | month_date_renewal | day_date_renewal | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2116.0 | 3.0 | 0.000000 | 234.405631 | 0.000000 | 2013-06-15 | 2016-06-15 | 2015-11-01 | 2015-06-23 | 0.000000 | 0.000000 | 0.0 | 1.334166 | 0.114481 | 0.098142 | 40.606701 | 1.0 | 0.000000 | 25.44 | 25.44 | 2 | 678.99 | 3 | 4.0 | 43.648 | 1 | 234.405631 | 0.212623 | 2013.0 | 6.0 | 15.0 | 2016.0 | 6.0 | 15.0 | 2015.0 | 11.0 | 1.0 | 2015.0 | 6.0 | 23.0 |
| 1 | 12004.0 | 3.0 | 68.264193 | 0.000000 | 0.000000 | 2009-08-21 | 2016-08-30 | 2009-08-21 | 2015-08-31 | 13.782235 | 0.000000 | 0.0 | 4.033609 | 0.145711 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 16.38 | 16.38 | 1 | 18.89 | 6 | 2.0 | 13.800 | 0 | 68.264193 | 0.145711 | 2009.0 | 8.0 | 21.0 | 2016.0 | 8.0 | 30.0 | 2009.0 | 8.0 | 21.0 | 2015.0 | 8.0 | 31.0 |
| 2 | 6713.0 | 3.0 | 23.323808 | 0.000000 | 0.000000 | 2010-04-16 | 2016-04-16 | 2010-04-16 | 2015-04-17 | 6.925316 | 0.000000 | 0.0 | 6.222540 | 0.165794 | 0.087899 | 44.311378 | 0.0 | 0.000000 | 28.60 | 28.60 | 1 | 6.60 | 6 | 2.0 | 13.856 | 0 | 23.323808 | 0.253693 | 2010.0 | 4.0 | 16.0 | 2016.0 | 4.0 | 16.0 | 2010.0 | 4.0 | 16.0 | 2015.0 | 4.0 | 17.0 |
| 3 | 10657.0 | 4.0 | 39.799497 | 0.000000 | 0.000000 | 2010-03-30 | 2016-03-30 | 2010-03-30 | 2015-03-31 | 15.493224 | 0.000000 | 0.0 | 4.453089 | 0.146694 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 30.22 | 30.22 | 1 | 25.46 | 6 | 2.0 | 13.200 | 0 | 39.799497 | 0.146694 | 2010.0 | 3.0 | 30.0 | 2016.0 | 3.0 | 30.0 | 2010.0 | 3.0 | 30.0 | 2015.0 | 3.0 | 31.0 |
| 4 | 1194.0 | 3.0 | 66.520673 | 0.000000 | 22.934690 | 2010-01-13 | 2016-03-07 | 2010-01-13 | 2015-03-09 | 21.112792 | 22.934690 | 0.0 | 11.477369 | 0.116900 | 0.100015 | 40.606701 | 0.0 | 7.233257 | 44.91 | 44.91 | 1 | 47.98 | 6 | 2.0 | 19.800 | 0 | 66.520673 | 0.216915 | 2010.0 | 1.0 | 13.0 | 2016.0 | 3.0 | 7.0 | 2010.0 | 1.0 | 13.0 | 2015.0 | 3.0 | 9.0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14591 | 9912.0 | 1.0 | 91.159201 | 0.000000 | 26.172505 | 2012-12-18 | 2016-12-18 | 2012-12-18 | 2015-12-21 | 28.862606 | 26.172505 | 0.0 | 11.478676 | 0.115237 | 0.100123 | 40.939027 | 0.0 | 8.187185 | 24.02 | 24.02 | 1 | 102.52 | 3 | 2.0 | 23.100 | 0 | 91.159201 | 0.215360 | 2012.0 | 12.0 | 18.0 | 2016.0 | 12.0 | 18.0 | 2012.0 | 12.0 | 18.0 | 2015.0 | 12.0 | 21.0 |
| 14592 | 1244.0 | 3.0 | 97.164808 | 0.000000 | 21.908902 | 2009-10-08 | 2016-10-08 | 2015-05-24 | 2015-10-09 | 31.368296 | 21.908902 | 0.0 | 11.493911 | 0.115237 | 0.100123 | 40.939027 | 0.0 | 6.854196 | 20.00 | 20.00 | 1 | 113.17 | 6 | 3.0 | 15.001 | 0 | 97.164808 | 0.215360 | 2009.0 | 10.0 | 8.0 | 2016.0 | 10.0 | 8.0 | 2015.0 | 5.0 | 24.0 | 2015.0 | 10.0 | 9.0 |
| 14593 | 4089.0 | 3.0 | 134.770175 | 55.099909 | 0.000000 | 2010-01-26 | 2017-01-26 | 2015-11-17 | 2016-01-27 | 51.612208 | 0.000000 | 0.0 | 4.043513 | 0.143575 | 0.000000 | 44.311378 | 1.0 | 0.000000 | 31.20 | 31.20 | 3 | 254.81 | 6 | 2.0 | 13.856 | 0 | 189.870084 | 0.143575 | 2010.0 | 1.0 | 26.0 | 2017.0 | 1.0 | 26.0 | 2015.0 | 11.0 | 17.0 | 2016.0 | 1.0 | 27.0 |
| 14594 | 11158.0 | 3.0 | 295.205014 | 0.000000 | 106.616134 | 2013-02-08 | 2016-02-08 | 2013-02-08 | 2015-02-09 | 26.689511 | 26.702060 | 0.0 | 12.075595 | 0.120372 | 0.103487 | 40.606701 | 0.0 | 8.474078 | 66.00 | 66.00 | 1 | 87.14 | 3 | 4.0 | 26.400 | 0 | 295.205014 | 0.223859 | 2013.0 | 2.0 | 8.0 | 2016.0 | 2.0 | 8.0 | 2013.0 | 2.0 | 8.0 | 2015.0 | 2.0 | 9.0 |
| 14595 | 1409.0 | 3.0 | 179.638526 | 218.952050 | 0.000000 | 2012-05-24 | 2016-05-08 | 2015-05-08 | 2014-05-26 | 68.176316 | 0.000000 | 0.0 | 4.309292 | 0.138305 | 0.000000 | 44.311378 | 1.0 | 0.000000 | 27.88 | 27.88 | 2 | 381.77 | 4 | 4.0 | 15.000 | 0 | 398.590575 | 0.138305 | 2012.0 | 5.0 | 24.0 | 2016.0 | 5.0 | 8.0 | 2015.0 | 5.0 | 8.0 | 2014.0 | 5.0 | 26.0 |
14596 rows × 40 columns
dfc.drop(columns=['date_activ',"date_end","date_modif_prod","date_renewal"],inplace=True)
dfc
| id | channel_sales | cons_12m | cons_gas_12m | cons_last_month | forecast_cons_12m | forecast_cons_year | forecast_discount_energy | forecast_meter_rent_12m | forecast_price_energy_off_peak | forecast_price_energy_peak | forecast_price_pow_off_peak | has_gas | imp_cons | margin_gross_pow_ele | margin_net_pow_ele | nb_prod_act | net_margin | num_years_antig | origin_up | pow_max | churn | total_cons_12m | fore_casted_price_energy_total | year_date_activ | month_date_activ | day_date_activ | year_date_end | month_date_end | day_date_end | year_date_modif_prod | month_date_modif_prod | day_date_modif_prod | year_date_renewal | month_date_renewal | day_date_renewal | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2116.0 | 3.0 | 0.000000 | 234.405631 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 1.334166 | 0.114481 | 0.098142 | 40.606701 | 1.0 | 0.000000 | 25.44 | 25.44 | 2 | 678.99 | 3 | 4.0 | 43.648 | 1 | 234.405631 | 0.212623 | 2013.0 | 6.0 | 15.0 | 2016.0 | 6.0 | 15.0 | 2015.0 | 11.0 | 1.0 | 2015.0 | 6.0 | 23.0 |
| 1 | 12004.0 | 3.0 | 68.264193 | 0.000000 | 0.000000 | 13.782235 | 0.000000 | 0.0 | 4.033609 | 0.145711 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 16.38 | 16.38 | 1 | 18.89 | 6 | 2.0 | 13.800 | 0 | 68.264193 | 0.145711 | 2009.0 | 8.0 | 21.0 | 2016.0 | 8.0 | 30.0 | 2009.0 | 8.0 | 21.0 | 2015.0 | 8.0 | 31.0 |
| 2 | 6713.0 | 3.0 | 23.323808 | 0.000000 | 0.000000 | 6.925316 | 0.000000 | 0.0 | 6.222540 | 0.165794 | 0.087899 | 44.311378 | 0.0 | 0.000000 | 28.60 | 28.60 | 1 | 6.60 | 6 | 2.0 | 13.856 | 0 | 23.323808 | 0.253693 | 2010.0 | 4.0 | 16.0 | 2016.0 | 4.0 | 16.0 | 2010.0 | 4.0 | 16.0 | 2015.0 | 4.0 | 17.0 |
| 3 | 10657.0 | 4.0 | 39.799497 | 0.000000 | 0.000000 | 15.493224 | 0.000000 | 0.0 | 4.453089 | 0.146694 | 0.000000 | 44.311378 | 0.0 | 0.000000 | 30.22 | 30.22 | 1 | 25.46 | 6 | 2.0 | 13.200 | 0 | 39.799497 | 0.146694 | 2010.0 | 3.0 | 30.0 | 2016.0 | 3.0 | 30.0 | 2010.0 | 3.0 | 30.0 | 2015.0 | 3.0 | 31.0 |
| 4 | 1194.0 | 3.0 | 66.520673 | 0.000000 | 22.934690 | 21.112792 | 22.934690 | 0.0 | 11.477369 | 0.116900 | 0.100015 | 40.606701 | 0.0 | 7.233257 | 44.91 | 44.91 | 1 | 47.98 | 6 | 2.0 | 19.800 | 0 | 66.520673 | 0.216915 | 2010.0 | 1.0 | 13.0 | 2016.0 | 3.0 | 7.0 | 2010.0 | 1.0 | 13.0 | 2015.0 | 3.0 | 9.0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 14591 | 9912.0 | 1.0 | 91.159201 | 0.000000 | 26.172505 | 28.862606 | 26.172505 | 0.0 | 11.478676 | 0.115237 | 0.100123 | 40.939027 | 0.0 | 8.187185 | 24.02 | 24.02 | 1 | 102.52 | 3 | 2.0 | 23.100 | 0 | 91.159201 | 0.215360 | 2012.0 | 12.0 | 18.0 | 2016.0 | 12.0 | 18.0 | 2012.0 | 12.0 | 18.0 | 2015.0 | 12.0 | 21.0 |
| 14592 | 1244.0 | 3.0 | 97.164808 | 0.000000 | 21.908902 | 31.368296 | 21.908902 | 0.0 | 11.493911 | 0.115237 | 0.100123 | 40.939027 | 0.0 | 6.854196 | 20.00 | 20.00 | 1 | 113.17 | 6 | 3.0 | 15.001 | 0 | 97.164808 | 0.215360 | 2009.0 | 10.0 | 8.0 | 2016.0 | 10.0 | 8.0 | 2015.0 | 5.0 | 24.0 | 2015.0 | 10.0 | 9.0 |
| 14593 | 4089.0 | 3.0 | 134.770175 | 55.099909 | 0.000000 | 51.612208 | 0.000000 | 0.0 | 4.043513 | 0.143575 | 0.000000 | 44.311378 | 1.0 | 0.000000 | 31.20 | 31.20 | 3 | 254.81 | 6 | 2.0 | 13.856 | 0 | 189.870084 | 0.143575 | 2010.0 | 1.0 | 26.0 | 2017.0 | 1.0 | 26.0 | 2015.0 | 11.0 | 17.0 | 2016.0 | 1.0 | 27.0 |
| 14594 | 11158.0 | 3.0 | 295.205014 | 0.000000 | 106.616134 | 26.689511 | 26.702060 | 0.0 | 12.075595 | 0.120372 | 0.103487 | 40.606701 | 0.0 | 8.474078 | 66.00 | 66.00 | 1 | 87.14 | 3 | 4.0 | 26.400 | 0 | 295.205014 | 0.223859 | 2013.0 | 2.0 | 8.0 | 2016.0 | 2.0 | 8.0 | 2013.0 | 2.0 | 8.0 | 2015.0 | 2.0 | 9.0 |
| 14595 | 1409.0 | 3.0 | 179.638526 | 218.952050 | 0.000000 | 68.176316 | 0.000000 | 0.0 | 4.309292 | 0.138305 | 0.000000 | 44.311378 | 1.0 | 0.000000 | 27.88 | 27.88 | 2 | 381.77 | 4 | 4.0 | 15.000 | 0 | 398.590575 | 0.138305 | 2012.0 | 5.0 | 24.0 | 2016.0 | 5.0 | 8.0 | 2015.0 | 5.0 | 8.0 | 2014.0 | 5.0 | 26.0 |
14596 rows × 36 columns
Now at this level we will move on to Feature normalization which is a process of scaling the data between 0 and 1 in order to facilitate the processing of the data for our modelling algorithm whilst creating our model and aim for better results. since this is numerical scaling we will only work on the numerical features only
def featureSelect(df_x):
return df_x.drop(['churn'], axis=1), df_x['churn']
def Normalization(X, scaler="minmax"):
if scaler.upper() == "STANDARD":
stand = StandardScaler()
X_s = stand.fit_transform(X)
return X_s
else:
minmax = MinMaxScaler()
X_mm = minmax.fit_transform(X)
return X_mm
X, y = featureSelect(dfc)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=0.2)
Now let's normalize our numerical features
dfc.columns
Index(['id', 'channel_sales', 'cons_12m', 'cons_gas_12m', 'cons_last_month',
'forecast_cons_12m', 'forecast_cons_year', 'forecast_discount_energy',
'forecast_meter_rent_12m', 'forecast_price_energy_off_peak',
'forecast_price_energy_peak', 'forecast_price_pow_off_peak', 'has_gas',
'imp_cons', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'nb_prod_act',
'net_margin', 'num_years_antig', 'origin_up', 'pow_max', 'churn',
'total_cons_12m', 'fore_casted_price_energy_total', 'year_date_activ',
'month_date_activ', 'day_date_activ', 'year_date_end', 'month_date_end',
'day_date_end', 'year_date_modif_prod', 'month_date_modif_prod',
'day_date_modif_prod', 'year_date_renewal', 'month_date_renewal',
'day_date_renewal'],
dtype='object')
col_to_norm =[ 'id', 'channel_sales', 'cons_12m', 'cons_gas_12m', 'cons_last_month',
'forecast_cons_12m', 'forecast_cons_year', 'forecast_discount_energy',
'forecast_meter_rent_12m', 'forecast_price_energy_off_peak',
'forecast_price_energy_peak', 'forecast_price_pow_off_peak', 'has_gas',
'imp_cons', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'nb_prod_act',
'net_margin', 'num_years_antig', 'origin_up', 'pow_max',
'total_cons_12m', 'fore_casted_price_energy_total', 'year_date_activ',
'month_date_activ', 'day_date_activ', 'year_date_end', 'month_date_end',
'day_date_end', 'year_date_modif_prod', 'month_date_modif_prod',
'day_date_modif_prod', 'year_date_renewal', 'month_date_renewal',
'day_date_renewal']
X_train[col_to_norm] = Normalization(X_train[col_to_norm])
# CatBoost model definition
catboost_model = CatBoostClassifier(iterations=1000,
custom_loss=['Accuracy'],
loss_function='Logloss'
)
train_pool = Pool(X_train,y_train)
# Fit CatBoost model
catboost_model.fit(train_pool,
plot=True)
MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))
Learning rate set to 0.029422 0: learn: 0.6664131 total: 24.5ms remaining: 24.5s 1: learn: 0.6414146 total: 34.6ms remaining: 17.3s 2: learn: 0.6179993 total: 44.6ms remaining: 14.8s 3: learn: 0.5970440 total: 54.1ms remaining: 13.5s 4: learn: 0.5773781 total: 63.8ms remaining: 12.7s 5: learn: 0.5585749 total: 73.6ms remaining: 12.2s 6: learn: 0.5409450 total: 83.6ms remaining: 11.9s 7: learn: 0.5247728 total: 93.4ms remaining: 11.6s 8: learn: 0.5100749 total: 103ms remaining: 11.3s 9: learn: 0.4963289 total: 112ms remaining: 11.1s 10: learn: 0.4834125 total: 121ms remaining: 10.9s 11: learn: 0.4711993 total: 130ms remaining: 10.7s 12: learn: 0.4599212 total: 139ms remaining: 10.6s 13: learn: 0.4495486 total: 149ms remaining: 10.5s 14: learn: 0.4396302 total: 158ms remaining: 10.4s 15: learn: 0.4308134 total: 167ms remaining: 10.3s 16: learn: 0.4219958 total: 176ms remaining: 10.2s 17: learn: 0.4146000 total: 185ms remaining: 10.1s 18: learn: 0.4077067 total: 195ms remaining: 10.1s 19: learn: 0.4010082 total: 205ms remaining: 10s 20: learn: 0.3949090 total: 214ms remaining: 9.98s 21: learn: 0.3887154 total: 223ms remaining: 9.92s 22: learn: 0.3835791 total: 232ms remaining: 9.85s 23: learn: 0.3782954 total: 241ms remaining: 9.8s 24: learn: 0.3734522 total: 250ms remaining: 9.75s 25: learn: 0.3689483 total: 259ms remaining: 9.71s 26: learn: 0.3648187 total: 269ms remaining: 9.68s 27: learn: 0.3608551 total: 278ms remaining: 9.65s 28: learn: 0.3575745 total: 287ms remaining: 9.61s 29: learn: 0.3539979 total: 296ms remaining: 9.57s 30: learn: 0.3508401 total: 305ms remaining: 9.53s 31: learn: 0.3476797 total: 314ms remaining: 9.5s 32: learn: 0.3447049 total: 327ms remaining: 9.59s 33: learn: 0.3423652 total: 336ms remaining: 9.56s 34: learn: 0.3396420 total: 346ms remaining: 9.53s 35: learn: 0.3371924 total: 355ms remaining: 9.5s 36: learn: 0.3352790 total: 364ms remaining: 9.47s 37: learn: 0.3334615 total: 373ms remaining: 9.44s 38: learn: 0.3314024 total: 383ms remaining: 9.44s 39: learn: 0.3293583 total: 393ms remaining: 9.42s 40: learn: 0.3279527 total: 399ms remaining: 9.34s 41: learn: 0.3265371 total: 408ms remaining: 9.3s 42: learn: 0.3245532 total: 417ms remaining: 9.28s 43: learn: 0.3230126 total: 426ms remaining: 9.25s 44: learn: 0.3218236 total: 434ms remaining: 9.22s 45: learn: 0.3205391 total: 443ms remaining: 9.18s 46: learn: 0.3193090 total: 451ms remaining: 9.15s 47: learn: 0.3180445 total: 460ms remaining: 9.12s 48: learn: 0.3171182 total: 468ms remaining: 9.09s 49: learn: 0.3161859 total: 477ms remaining: 9.06s 50: learn: 0.3152460 total: 486ms remaining: 9.04s 51: learn: 0.3144157 total: 494ms remaining: 9s 52: learn: 0.3130873 total: 502ms remaining: 8.98s 53: learn: 0.3119889 total: 511ms remaining: 8.96s 54: learn: 0.3113893 total: 520ms remaining: 8.93s 55: learn: 0.3103089 total: 528ms remaining: 8.9s 56: learn: 0.3094325 total: 537ms remaining: 8.88s 57: learn: 0.3089100 total: 545ms remaining: 8.86s 58: learn: 0.3083399 total: 554ms remaining: 8.83s 59: learn: 0.3076193 total: 562ms remaining: 8.81s 60: learn: 0.3069264 total: 571ms remaining: 8.79s 61: learn: 0.3060289 total: 581ms remaining: 8.79s 62: learn: 0.3052432 total: 591ms remaining: 8.78s 63: learn: 0.3046502 total: 599ms remaining: 8.76s 64: learn: 0.3038698 total: 609ms remaining: 8.76s 65: learn: 0.3032191 total: 618ms remaining: 8.74s 66: learn: 0.3024843 total: 627ms remaining: 8.73s 67: learn: 0.3017232 total: 635ms remaining: 8.71s 68: learn: 0.3009435 total: 644ms remaining: 8.69s 69: learn: 0.3006028 total: 653ms remaining: 8.67s 70: learn: 0.3002238 total: 661ms remaining: 8.65s 71: learn: 0.2995808 total: 670ms remaining: 8.63s 72: learn: 0.2990948 total: 679ms remaining: 8.62s 73: learn: 0.2986692 total: 687ms remaining: 8.6s 74: learn: 0.2983860 total: 696ms remaining: 8.58s 75: learn: 0.2980584 total: 704ms remaining: 8.56s 76: learn: 0.2975359 total: 713ms remaining: 8.54s 77: learn: 0.2971115 total: 721ms remaining: 8.53s 78: learn: 0.2967524 total: 730ms remaining: 8.51s 79: learn: 0.2963365 total: 738ms remaining: 8.49s 80: learn: 0.2959447 total: 747ms remaining: 8.48s 81: learn: 0.2957088 total: 756ms remaining: 8.46s 82: learn: 0.2950501 total: 764ms remaining: 8.44s 83: learn: 0.2946290 total: 774ms remaining: 8.44s 84: learn: 0.2942985 total: 784ms remaining: 8.43s 85: learn: 0.2938288 total: 793ms remaining: 8.43s 86: learn: 0.2934363 total: 803ms remaining: 8.43s 87: learn: 0.2931491 total: 813ms remaining: 8.42s 88: learn: 0.2927286 total: 823ms remaining: 8.42s 89: learn: 0.2924480 total: 832ms remaining: 8.41s 90: learn: 0.2921401 total: 841ms remaining: 8.4s 91: learn: 0.2916834 total: 851ms remaining: 8.39s 92: learn: 0.2914755 total: 861ms remaining: 8.39s 93: learn: 0.2911294 total: 870ms remaining: 8.39s 94: learn: 0.2906807 total: 879ms remaining: 8.38s 95: learn: 0.2902997 total: 889ms remaining: 8.37s 96: learn: 0.2898418 total: 898ms remaining: 8.36s 97: learn: 0.2896066 total: 906ms remaining: 8.34s 98: learn: 0.2892964 total: 915ms remaining: 8.33s 99: learn: 0.2890155 total: 924ms remaining: 8.31s 100: learn: 0.2886767 total: 933ms remaining: 8.3s 101: learn: 0.2884637 total: 942ms remaining: 8.29s 102: learn: 0.2881466 total: 950ms remaining: 8.28s 103: learn: 0.2877601 total: 961ms remaining: 8.28s 104: learn: 0.2875172 total: 970ms remaining: 8.27s 105: learn: 0.2872336 total: 982ms remaining: 8.28s 106: learn: 0.2869034 total: 994ms remaining: 8.29s 107: learn: 0.2866872 total: 1s remaining: 8.29s 108: learn: 0.2864231 total: 1.01s remaining: 8.28s 109: learn: 0.2861775 total: 1.02s remaining: 8.28s 110: learn: 0.2859996 total: 1.03s remaining: 8.27s 111: learn: 0.2856743 total: 1.04s remaining: 8.26s 112: learn: 0.2854096 total: 1.05s remaining: 8.26s 113: learn: 0.2851712 total: 1.06s remaining: 8.25s 114: learn: 0.2849008 total: 1.07s remaining: 8.24s 115: learn: 0.2846426 total: 1.08s remaining: 8.23s 116: learn: 0.2844306 total: 1.09s remaining: 8.23s 117: learn: 0.2843001 total: 1.1s remaining: 8.23s 118: learn: 0.2840639 total: 1.11s remaining: 8.22s 119: learn: 0.2837632 total: 1.12s remaining: 8.21s 120: learn: 0.2835368 total: 1.13s remaining: 8.21s 121: learn: 0.2831963 total: 1.14s remaining: 8.22s 122: learn: 0.2828464 total: 1.15s remaining: 8.23s 123: learn: 0.2826419 total: 1.17s remaining: 8.23s 124: learn: 0.2822731 total: 1.18s remaining: 8.24s 125: learn: 0.2820163 total: 1.19s remaining: 8.24s 126: learn: 0.2818654 total: 1.2s remaining: 8.25s 127: learn: 0.2814828 total: 1.21s remaining: 8.26s 128: learn: 0.2810810 total: 1.22s remaining: 8.26s 129: learn: 0.2808961 total: 1.23s remaining: 8.26s 130: learn: 0.2806967 total: 1.24s remaining: 8.26s 131: learn: 0.2805058 total: 1.27s remaining: 8.35s 132: learn: 0.2800644 total: 1.28s remaining: 8.33s 133: learn: 0.2798502 total: 1.29s remaining: 8.32s 134: learn: 0.2796590 total: 1.3s remaining: 8.31s 135: learn: 0.2794374 total: 1.3s remaining: 8.3s 136: learn: 0.2792255 total: 1.31s remaining: 8.28s 137: learn: 0.2790797 total: 1.32s remaining: 8.27s 138: learn: 0.2788662 total: 1.33s remaining: 8.25s 139: learn: 0.2785874 total: 1.34s remaining: 8.24s 140: learn: 0.2783416 total: 1.35s remaining: 8.24s 141: learn: 0.2781221 total: 1.36s remaining: 8.22s 142: learn: 0.2779112 total: 1.37s remaining: 8.21s 143: learn: 0.2777904 total: 1.38s remaining: 8.2s 144: learn: 0.2775212 total: 1.39s remaining: 8.18s 145: learn: 0.2773650 total: 1.4s remaining: 8.18s 146: learn: 0.2770180 total: 1.41s remaining: 8.16s 147: learn: 0.2769109 total: 1.42s remaining: 8.15s 148: learn: 0.2767026 total: 1.43s remaining: 8.14s 149: learn: 0.2764504 total: 1.44s remaining: 8.14s 150: learn: 0.2763071 total: 1.45s remaining: 8.13s 151: learn: 0.2760494 total: 1.46s remaining: 8.12s 152: learn: 0.2759079 total: 1.47s remaining: 8.11s 153: learn: 0.2757481 total: 1.48s remaining: 8.1s 154: learn: 0.2755830 total: 1.48s remaining: 8.09s 155: learn: 0.2754154 total: 1.5s remaining: 8.09s 156: learn: 0.2752234 total: 1.5s remaining: 8.08s 157: learn: 0.2749132 total: 1.51s remaining: 8.07s 158: learn: 0.2746735 total: 1.52s remaining: 8.07s 159: learn: 0.2744301 total: 1.53s remaining: 8.06s 160: learn: 0.2743006 total: 1.54s remaining: 8.05s 161: learn: 0.2741000 total: 1.55s remaining: 8.04s 162: learn: 0.2739301 total: 1.56s remaining: 8.03s 163: learn: 0.2737578 total: 1.57s remaining: 8.03s 164: learn: 0.2735936 total: 1.58s remaining: 8.02s 165: learn: 0.2734412 total: 1.59s remaining: 8s 166: learn: 0.2732508 total: 1.6s remaining: 7.99s 167: learn: 0.2730580 total: 1.61s remaining: 7.98s 168: learn: 0.2727310 total: 1.62s remaining: 7.97s 169: learn: 0.2726394 total: 1.63s remaining: 7.95s 170: learn: 0.2724290 total: 1.64s remaining: 7.94s 171: learn: 0.2722454 total: 1.65s remaining: 7.92s 172: learn: 0.2720432 total: 1.65s remaining: 7.91s 173: learn: 0.2717729 total: 1.66s remaining: 7.89s 174: learn: 0.2716136 total: 1.67s remaining: 7.88s 175: learn: 0.2713549 total: 1.68s remaining: 7.87s 176: learn: 0.2711867 total: 1.69s remaining: 7.85s 177: learn: 0.2711078 total: 1.7s remaining: 7.84s 178: learn: 0.2708572 total: 1.71s remaining: 7.82s 179: learn: 0.2706531 total: 1.71s remaining: 7.81s 180: learn: 0.2704917 total: 1.72s remaining: 7.8s 181: learn: 0.2703816 total: 1.73s remaining: 7.79s 182: learn: 0.2701849 total: 1.74s remaining: 7.78s 183: learn: 0.2699951 total: 1.75s remaining: 7.77s 184: learn: 0.2698861 total: 1.76s remaining: 7.76s 185: learn: 0.2697475 total: 1.77s remaining: 7.74s 186: learn: 0.2696184 total: 1.78s remaining: 7.73s 187: learn: 0.2693990 total: 1.79s remaining: 7.72s 188: learn: 0.2692326 total: 1.79s remaining: 7.71s 189: learn: 0.2690572 total: 1.8s remaining: 7.7s 190: learn: 0.2689016 total: 1.81s remaining: 7.69s 191: learn: 0.2687646 total: 1.82s remaining: 7.67s 192: learn: 0.2686327 total: 1.83s remaining: 7.66s 193: learn: 0.2684841 total: 1.84s remaining: 7.66s 194: learn: 0.2682829 total: 1.85s remaining: 7.66s 195: learn: 0.2680565 total: 1.86s remaining: 7.65s 196: learn: 0.2679174 total: 1.87s remaining: 7.63s 197: learn: 0.2677696 total: 1.88s remaining: 7.62s 198: learn: 0.2676374 total: 1.89s remaining: 7.61s 199: learn: 0.2673682 total: 1.9s remaining: 7.59s 200: learn: 0.2671539 total: 1.91s remaining: 7.58s 201: learn: 0.2669596 total: 1.92s remaining: 7.57s 202: learn: 0.2666639 total: 1.93s remaining: 7.56s 203: learn: 0.2665157 total: 1.94s remaining: 7.57s 204: learn: 0.2662877 total: 1.95s remaining: 7.55s 205: learn: 0.2660466 total: 1.96s remaining: 7.54s 206: learn: 0.2658888 total: 1.97s remaining: 7.53s 207: learn: 0.2657540 total: 1.97s remaining: 7.52s 208: learn: 0.2655301 total: 1.98s remaining: 7.51s 209: learn: 0.2652508 total: 1.99s remaining: 7.5s 210: learn: 0.2651292 total: 2s remaining: 7.49s 211: learn: 0.2649740 total: 2.01s remaining: 7.48s 212: learn: 0.2647735 total: 2.02s remaining: 7.47s 213: learn: 0.2646439 total: 2.03s remaining: 7.46s 214: learn: 0.2644881 total: 2.04s remaining: 7.45s 215: learn: 0.2643794 total: 2.05s remaining: 7.43s 216: learn: 0.2641960 total: 2.06s remaining: 7.42s 217: learn: 0.2640605 total: 2.07s remaining: 7.41s 218: learn: 0.2639739 total: 2.08s remaining: 7.4s 219: learn: 0.2638678 total: 2.08s remaining: 7.39s 220: learn: 0.2636981 total: 2.09s remaining: 7.38s 221: learn: 0.2634079 total: 2.1s remaining: 7.37s 222: learn: 0.2632822 total: 2.11s remaining: 7.36s 223: learn: 0.2630615 total: 2.12s remaining: 7.35s 224: learn: 0.2629140 total: 2.13s remaining: 7.34s 225: learn: 0.2627328 total: 2.14s remaining: 7.33s 226: learn: 0.2625235 total: 2.15s remaining: 7.32s 227: learn: 0.2623986 total: 2.16s remaining: 7.31s 228: learn: 0.2622749 total: 2.17s remaining: 7.3s 229: learn: 0.2621512 total: 2.18s remaining: 7.29s 230: learn: 0.2619507 total: 2.19s remaining: 7.28s 231: learn: 0.2617923 total: 2.19s remaining: 7.27s 232: learn: 0.2616519 total: 2.2s remaining: 7.26s 233: learn: 0.2615141 total: 2.21s remaining: 7.25s 234: learn: 0.2612284 total: 2.22s remaining: 7.24s 235: learn: 0.2610344 total: 2.23s remaining: 7.23s 236: learn: 0.2608619 total: 2.24s remaining: 7.22s 237: learn: 0.2606468 total: 2.25s remaining: 7.21s 238: learn: 0.2605100 total: 2.26s remaining: 7.2s 239: learn: 0.2603697 total: 2.27s remaining: 7.19s 240: learn: 0.2602281 total: 2.28s remaining: 7.19s 241: learn: 0.2600057 total: 2.29s remaining: 7.18s 242: learn: 0.2597499 total: 2.3s remaining: 7.17s 243: learn: 0.2595792 total: 2.31s remaining: 7.16s 244: learn: 0.2594363 total: 2.32s remaining: 7.16s 245: learn: 0.2592328 total: 2.33s remaining: 7.15s 246: learn: 0.2590418 total: 2.34s remaining: 7.14s 247: learn: 0.2589117 total: 2.35s remaining: 7.13s 248: learn: 0.2586713 total: 2.36s remaining: 7.12s 249: learn: 0.2584695 total: 2.37s remaining: 7.11s 250: learn: 0.2583647 total: 2.38s remaining: 7.1s 251: learn: 0.2581943 total: 2.39s remaining: 7.09s 252: learn: 0.2580364 total: 2.4s remaining: 7.08s 253: learn: 0.2578626 total: 2.41s remaining: 7.08s 254: learn: 0.2576786 total: 2.42s remaining: 7.07s 255: learn: 0.2575401 total: 2.43s remaining: 7.06s 256: learn: 0.2574000 total: 2.44s remaining: 7.05s 257: learn: 0.2572045 total: 2.45s remaining: 7.04s 258: learn: 0.2569965 total: 2.46s remaining: 7.03s 259: learn: 0.2567792 total: 2.47s remaining: 7.02s 260: learn: 0.2566083 total: 2.48s remaining: 7.01s 261: learn: 0.2564768 total: 2.48s remaining: 7s 262: learn: 0.2562005 total: 2.49s remaining: 6.99s 263: learn: 0.2560785 total: 2.5s remaining: 6.98s 264: learn: 0.2559453 total: 2.51s remaining: 6.97s 265: learn: 0.2558139 total: 2.52s remaining: 6.96s 266: learn: 0.2556868 total: 2.53s remaining: 6.95s 267: learn: 0.2555261 total: 2.54s remaining: 6.93s 268: learn: 0.2554123 total: 2.55s remaining: 6.92s 269: learn: 0.2552253 total: 2.56s remaining: 6.91s 270: learn: 0.2550628 total: 2.57s remaining: 6.9s 271: learn: 0.2548424 total: 2.58s remaining: 6.89s 272: learn: 0.2546929 total: 2.58s remaining: 6.88s 273: learn: 0.2545951 total: 2.59s remaining: 6.87s 274: learn: 0.2544361 total: 2.6s remaining: 6.86s 275: learn: 0.2542333 total: 2.61s remaining: 6.85s 276: learn: 0.2540391 total: 2.62s remaining: 6.84s 277: learn: 0.2538792 total: 2.63s remaining: 6.83s 278: learn: 0.2537358 total: 2.64s remaining: 6.81s 279: learn: 0.2535783 total: 2.65s remaining: 6.8s 280: learn: 0.2533659 total: 2.65s remaining: 6.79s 281: learn: 0.2531955 total: 2.66s remaining: 6.78s 282: learn: 0.2529472 total: 2.67s remaining: 6.77s 283: learn: 0.2528308 total: 2.68s remaining: 6.76s 284: learn: 0.2527410 total: 2.69s remaining: 6.75s 285: learn: 0.2525930 total: 2.7s remaining: 6.74s 286: learn: 0.2524116 total: 2.71s remaining: 6.73s 287: learn: 0.2522303 total: 2.72s remaining: 6.72s 288: learn: 0.2521253 total: 2.73s remaining: 6.71s 289: learn: 0.2520249 total: 2.73s remaining: 6.7s 290: learn: 0.2519010 total: 2.74s remaining: 6.68s 291: learn: 0.2517152 total: 2.75s remaining: 6.67s 292: learn: 0.2515781 total: 2.76s remaining: 6.66s 293: learn: 0.2514403 total: 2.77s remaining: 6.65s 294: learn: 0.2513111 total: 2.78s remaining: 6.64s 295: learn: 0.2511367 total: 2.79s remaining: 6.63s 296: learn: 0.2509403 total: 2.79s remaining: 6.62s 297: learn: 0.2507840 total: 2.8s remaining: 6.6s 298: learn: 0.2506522 total: 2.81s remaining: 6.59s 299: learn: 0.2505631 total: 2.82s remaining: 6.58s 300: learn: 0.2504116 total: 2.83s remaining: 6.57s 301: learn: 0.2502512 total: 2.84s remaining: 6.56s 302: learn: 0.2501482 total: 2.85s remaining: 6.55s 303: learn: 0.2499423 total: 2.86s remaining: 6.54s 304: learn: 0.2498554 total: 2.87s remaining: 6.53s 305: learn: 0.2496795 total: 2.87s remaining: 6.52s 306: learn: 0.2494892 total: 2.88s remaining: 6.51s 307: learn: 0.2493504 total: 2.89s remaining: 6.5s 308: learn: 0.2491925 total: 2.9s remaining: 6.49s 309: learn: 0.2490176 total: 2.91s remaining: 6.48s 310: learn: 0.2488403 total: 2.92s remaining: 6.47s 311: learn: 0.2486559 total: 2.93s remaining: 6.46s 312: learn: 0.2484295 total: 2.94s remaining: 6.45s 313: learn: 0.2482814 total: 2.94s remaining: 6.43s 314: learn: 0.2481329 total: 2.95s remaining: 6.42s 315: learn: 0.2479586 total: 2.96s remaining: 6.41s 316: learn: 0.2477446 total: 2.97s remaining: 6.4s 317: learn: 0.2475861 total: 2.99s remaining: 6.41s 318: learn: 0.2474691 total: 3s remaining: 6.4s 319: learn: 0.2472611 total: 3.01s remaining: 6.39s 320: learn: 0.2470600 total: 3.02s remaining: 6.38s 321: learn: 0.2469524 total: 3.02s remaining: 6.37s 322: learn: 0.2468045 total: 3.03s remaining: 6.36s 323: learn: 0.2466668 total: 3.04s remaining: 6.34s 324: learn: 0.2465277 total: 3.05s remaining: 6.33s 325: learn: 0.2463919 total: 3.06s remaining: 6.32s 326: learn: 0.2462150 total: 3.07s remaining: 6.31s 327: learn: 0.2460259 total: 3.08s remaining: 6.3s 328: learn: 0.2458504 total: 3.08s remaining: 6.29s 329: learn: 0.2456471 total: 3.09s remaining: 6.28s 330: learn: 0.2455038 total: 3.1s remaining: 6.27s 331: learn: 0.2452747 total: 3.11s remaining: 6.26s 332: learn: 0.2451686 total: 3.12s remaining: 6.25s 333: learn: 0.2450011 total: 3.13s remaining: 6.24s 334: learn: 0.2448277 total: 3.14s remaining: 6.23s 335: learn: 0.2446500 total: 3.15s remaining: 6.22s 336: learn: 0.2445633 total: 3.15s remaining: 6.21s 337: learn: 0.2444241 total: 3.16s remaining: 6.2s 338: learn: 0.2442523 total: 3.17s remaining: 6.19s 339: learn: 0.2441404 total: 3.18s remaining: 6.17s 340: learn: 0.2440518 total: 3.19s remaining: 6.16s 341: learn: 0.2439003 total: 3.2s remaining: 6.15s 342: learn: 0.2437246 total: 3.21s remaining: 6.14s 343: learn: 0.2435453 total: 3.21s remaining: 6.13s 344: learn: 0.2434113 total: 3.22s remaining: 6.12s 345: learn: 0.2432653 total: 3.23s remaining: 6.11s 346: learn: 0.2431352 total: 3.24s remaining: 6.1s 347: learn: 0.2428993 total: 3.25s remaining: 6.09s 348: learn: 0.2427887 total: 3.26s remaining: 6.08s 349: learn: 0.2426357 total: 3.27s remaining: 6.07s 350: learn: 0.2425497 total: 3.28s remaining: 6.06s 351: learn: 0.2424273 total: 3.29s remaining: 6.05s 352: learn: 0.2422452 total: 3.29s remaining: 6.04s 353: learn: 0.2420839 total: 3.3s remaining: 6.03s 354: learn: 0.2419085 total: 3.31s remaining: 6.01s 355: learn: 0.2418314 total: 3.32s remaining: 6s 356: learn: 0.2416975 total: 3.33s remaining: 5.99s 357: learn: 0.2414923 total: 3.34s remaining: 5.98s 358: learn: 0.2413327 total: 3.35s remaining: 5.97s 359: learn: 0.2411316 total: 3.35s remaining: 5.96s 360: learn: 0.2410124 total: 3.36s remaining: 5.95s 361: learn: 0.2409290 total: 3.37s remaining: 5.94s 362: learn: 0.2408180 total: 3.38s remaining: 5.93s 363: learn: 0.2406604 total: 3.39s remaining: 5.92s 364: learn: 0.2405462 total: 3.4s remaining: 5.91s 365: learn: 0.2403497 total: 3.41s remaining: 5.9s 366: learn: 0.2402186 total: 3.42s remaining: 5.89s 367: learn: 0.2400975 total: 3.42s remaining: 5.88s 368: learn: 0.2400155 total: 3.43s remaining: 5.87s 369: learn: 0.2399089 total: 3.44s remaining: 5.86s 370: learn: 0.2397497 total: 3.45s remaining: 5.85s 371: learn: 0.2395609 total: 3.46s remaining: 5.84s 372: learn: 0.2394712 total: 3.47s remaining: 5.83s 373: learn: 0.2392795 total: 3.48s remaining: 5.83s 374: learn: 0.2391077 total: 3.49s remaining: 5.82s 375: learn: 0.2389591 total: 3.5s remaining: 5.8s 376: learn: 0.2387975 total: 3.51s remaining: 5.79s 377: learn: 0.2386475 total: 3.51s remaining: 5.78s 378: learn: 0.2384583 total: 3.52s remaining: 5.77s 379: learn: 0.2383246 total: 3.53s remaining: 5.76s 380: learn: 0.2382125 total: 3.54s remaining: 5.75s 381: learn: 0.2380795 total: 3.55s remaining: 5.74s 382: learn: 0.2379723 total: 3.56s remaining: 5.73s 383: learn: 0.2378424 total: 3.57s remaining: 5.72s 384: learn: 0.2376807 total: 3.58s remaining: 5.71s 385: learn: 0.2375289 total: 3.58s remaining: 5.7s 386: learn: 0.2373615 total: 3.59s remaining: 5.69s 387: learn: 0.2372254 total: 3.6s remaining: 5.68s 388: learn: 0.2370385 total: 3.61s remaining: 5.67s 389: learn: 0.2369145 total: 3.62s remaining: 5.66s 390: learn: 0.2367452 total: 3.63s remaining: 5.65s 391: learn: 0.2366130 total: 3.64s remaining: 5.64s 392: learn: 0.2364555 total: 3.65s remaining: 5.63s 393: learn: 0.2363208 total: 3.66s remaining: 5.62s 394: learn: 0.2361514 total: 3.67s remaining: 5.62s 395: learn: 0.2360220 total: 3.67s remaining: 5.61s 396: learn: 0.2358756 total: 3.68s remaining: 5.6s 397: learn: 0.2357259 total: 3.69s remaining: 5.59s 398: learn: 0.2355434 total: 3.7s remaining: 5.58s 399: learn: 0.2353784 total: 3.71s remaining: 5.57s 400: learn: 0.2352491 total: 3.72s remaining: 5.56s 401: learn: 0.2351260 total: 3.73s remaining: 5.55s 402: learn: 0.2349609 total: 3.74s remaining: 5.54s 403: learn: 0.2347994 total: 3.75s remaining: 5.53s 404: learn: 0.2346916 total: 3.75s remaining: 5.52s 405: learn: 0.2345687 total: 3.76s remaining: 5.5s 406: learn: 0.2344303 total: 3.77s remaining: 5.5s 407: learn: 0.2343390 total: 3.78s remaining: 5.48s 408: learn: 0.2342180 total: 3.79s remaining: 5.47s 409: learn: 0.2341418 total: 3.8s remaining: 5.46s 410: learn: 0.2340184 total: 3.81s remaining: 5.45s 411: learn: 0.2338744 total: 3.81s remaining: 5.45s 412: learn: 0.2337061 total: 3.82s remaining: 5.44s 413: learn: 0.2335720 total: 3.83s remaining: 5.43s 414: learn: 0.2334729 total: 3.84s remaining: 5.42s 415: learn: 0.2333335 total: 3.85s remaining: 5.41s 416: learn: 0.2332302 total: 3.86s remaining: 5.4s 417: learn: 0.2330398 total: 3.87s remaining: 5.38s 418: learn: 0.2329418 total: 3.88s remaining: 5.38s 419: learn: 0.2327961 total: 3.88s remaining: 5.37s 420: learn: 0.2326611 total: 3.89s remaining: 5.35s 421: learn: 0.2325602 total: 3.9s remaining: 5.34s 422: learn: 0.2324539 total: 3.91s remaining: 5.33s 423: learn: 0.2322812 total: 3.92s remaining: 5.32s 424: learn: 0.2321869 total: 3.93s remaining: 5.31s 425: learn: 0.2320569 total: 3.94s remaining: 5.3s 426: learn: 0.2319463 total: 3.95s remaining: 5.29s 427: learn: 0.2318125 total: 3.95s remaining: 5.29s 428: learn: 0.2317104 total: 3.96s remaining: 5.28s 429: learn: 0.2315924 total: 3.97s remaining: 5.26s 430: learn: 0.2313938 total: 3.98s remaining: 5.25s 431: learn: 0.2312348 total: 4s remaining: 5.26s 432: learn: 0.2311276 total: 4.01s remaining: 5.25s 433: learn: 0.2309550 total: 4.02s remaining: 5.24s 434: learn: 0.2308841 total: 4.03s remaining: 5.23s 435: learn: 0.2307610 total: 4.04s remaining: 5.22s 436: learn: 0.2305674 total: 4.04s remaining: 5.21s 437: learn: 0.2305346 total: 4.05s remaining: 5.2s 438: learn: 0.2304477 total: 4.06s remaining: 5.19s 439: learn: 0.2302471 total: 4.07s remaining: 5.18s 440: learn: 0.2301297 total: 4.08s remaining: 5.17s 441: learn: 0.2299658 total: 4.09s remaining: 5.16s 442: learn: 0.2298229 total: 4.09s remaining: 5.15s 443: learn: 0.2296981 total: 4.1s remaining: 5.14s 444: learn: 0.2295897 total: 4.11s remaining: 5.13s 445: learn: 0.2294617 total: 4.12s remaining: 5.12s 446: learn: 0.2292636 total: 4.13s remaining: 5.11s 447: learn: 0.2290406 total: 4.14s remaining: 5.1s 448: learn: 0.2289086 total: 4.15s remaining: 5.09s 449: learn: 0.2287827 total: 4.16s remaining: 5.08s 450: learn: 0.2286557 total: 4.16s remaining: 5.07s 451: learn: 0.2285447 total: 4.17s remaining: 5.06s 452: learn: 0.2284336 total: 4.18s remaining: 5.05s 453: learn: 0.2283532 total: 4.19s remaining: 5.04s 454: learn: 0.2282081 total: 4.2s remaining: 5.03s 455: learn: 0.2281120 total: 4.21s remaining: 5.02s 456: learn: 0.2280325 total: 4.22s remaining: 5.01s 457: learn: 0.2279067 total: 4.23s remaining: 5s 458: learn: 0.2277507 total: 4.24s remaining: 5s 459: learn: 0.2276237 total: 4.25s remaining: 4.99s 460: learn: 0.2274905 total: 4.26s remaining: 4.99s 461: learn: 0.2274061 total: 4.27s remaining: 4.98s 462: learn: 0.2272695 total: 4.28s remaining: 4.97s 463: learn: 0.2271701 total: 4.29s remaining: 4.96s 464: learn: 0.2270513 total: 4.3s remaining: 4.95s 465: learn: 0.2269151 total: 4.31s remaining: 4.94s 466: learn: 0.2267380 total: 4.33s remaining: 4.94s 467: learn: 0.2265848 total: 4.34s remaining: 4.93s 468: learn: 0.2264528 total: 4.34s remaining: 4.92s 469: learn: 0.2263277 total: 4.35s remaining: 4.91s 470: learn: 0.2261721 total: 4.36s remaining: 4.9s 471: learn: 0.2260207 total: 4.37s remaining: 4.89s 472: learn: 0.2259198 total: 4.38s remaining: 4.88s 473: learn: 0.2257243 total: 4.39s remaining: 4.87s 474: learn: 0.2255546 total: 4.4s remaining: 4.86s 475: learn: 0.2254678 total: 4.41s remaining: 4.85s 476: learn: 0.2253051 total: 4.42s remaining: 4.84s 477: learn: 0.2251335 total: 4.43s remaining: 4.83s 478: learn: 0.2249363 total: 4.44s remaining: 4.82s 479: learn: 0.2248331 total: 4.45s remaining: 4.82s 480: learn: 0.2247060 total: 4.46s remaining: 4.81s 481: learn: 0.2246219 total: 4.46s remaining: 4.8s 482: learn: 0.2245236 total: 4.47s remaining: 4.79s 483: learn: 0.2243928 total: 4.48s remaining: 4.78s 484: learn: 0.2243090 total: 4.49s remaining: 4.77s 485: learn: 0.2242068 total: 4.5s remaining: 4.76s 486: learn: 0.2241071 total: 4.51s remaining: 4.75s 487: learn: 0.2239461 total: 4.52s remaining: 4.74s 488: learn: 0.2238371 total: 4.53s remaining: 4.73s 489: learn: 0.2237369 total: 4.54s remaining: 4.72s 490: learn: 0.2236514 total: 4.54s remaining: 4.71s 491: learn: 0.2235400 total: 4.55s remaining: 4.7s 492: learn: 0.2234295 total: 4.56s remaining: 4.69s 493: learn: 0.2233072 total: 4.57s remaining: 4.68s 494: learn: 0.2232001 total: 4.58s remaining: 4.67s 495: learn: 0.2230219 total: 4.59s remaining: 4.66s 496: learn: 0.2229117 total: 4.59s remaining: 4.65s 497: learn: 0.2227602 total: 4.6s remaining: 4.64s 498: learn: 0.2226179 total: 4.61s remaining: 4.63s 499: learn: 0.2225213 total: 4.62s remaining: 4.62s 500: learn: 0.2224048 total: 4.63s remaining: 4.61s 501: learn: 0.2222955 total: 4.64s remaining: 4.6s 502: learn: 0.2221325 total: 4.65s remaining: 4.59s 503: learn: 0.2220017 total: 4.66s remaining: 4.58s 504: learn: 0.2218458 total: 4.67s remaining: 4.58s 505: learn: 0.2217328 total: 4.68s remaining: 4.57s 506: learn: 0.2216059 total: 4.69s remaining: 4.56s 507: learn: 0.2214840 total: 4.7s remaining: 4.55s 508: learn: 0.2213401 total: 4.7s remaining: 4.54s 509: learn: 0.2212241 total: 4.71s remaining: 4.53s 510: learn: 0.2210771 total: 4.72s remaining: 4.52s 511: learn: 0.2209674 total: 4.73s remaining: 4.51s 512: learn: 0.2208301 total: 4.74s remaining: 4.5s 513: learn: 0.2207277 total: 4.75s remaining: 4.49s 514: learn: 0.2206409 total: 4.76s remaining: 4.48s 515: learn: 0.2205040 total: 4.77s remaining: 4.47s 516: learn: 0.2203624 total: 4.78s remaining: 4.46s 517: learn: 0.2202521 total: 4.79s remaining: 4.45s 518: learn: 0.2201437 total: 4.79s remaining: 4.44s 519: learn: 0.2200607 total: 4.8s remaining: 4.43s 520: learn: 0.2198786 total: 4.81s remaining: 4.42s 521: learn: 0.2197322 total: 4.82s remaining: 4.42s 522: learn: 0.2195702 total: 4.83s remaining: 4.41s 523: learn: 0.2194651 total: 4.84s remaining: 4.4s 524: learn: 0.2193371 total: 4.85s remaining: 4.39s 525: learn: 0.2192634 total: 4.86s remaining: 4.38s 526: learn: 0.2191587 total: 4.87s remaining: 4.37s 527: learn: 0.2190038 total: 4.87s remaining: 4.36s 528: learn: 0.2188734 total: 4.88s remaining: 4.35s 529: learn: 0.2187285 total: 4.89s remaining: 4.34s 530: learn: 0.2186335 total: 4.9s remaining: 4.33s 531: learn: 0.2184712 total: 4.91s remaining: 4.32s 532: learn: 0.2183729 total: 4.92s remaining: 4.31s 533: learn: 0.2182708 total: 4.92s remaining: 4.3s 534: learn: 0.2181324 total: 4.93s remaining: 4.29s 535: learn: 0.2179661 total: 4.94s remaining: 4.28s 536: learn: 0.2178662 total: 4.95s remaining: 4.27s 537: learn: 0.2177851 total: 4.96s remaining: 4.26s 538: learn: 0.2176486 total: 4.97s remaining: 4.25s 539: learn: 0.2175338 total: 4.98s remaining: 4.24s 540: learn: 0.2173977 total: 4.98s remaining: 4.23s 541: learn: 0.2172046 total: 4.99s remaining: 4.22s 542: learn: 0.2170623 total: 5.01s remaining: 4.22s 543: learn: 0.2169273 total: 5.03s remaining: 4.21s 544: learn: 0.2168617 total: 5.03s remaining: 4.2s 545: learn: 0.2167891 total: 5.04s remaining: 4.19s 546: learn: 0.2166816 total: 5.05s remaining: 4.18s 547: learn: 0.2165086 total: 5.06s remaining: 4.17s 548: learn: 0.2164122 total: 5.07s remaining: 4.16s 549: learn: 0.2162648 total: 5.08s remaining: 4.15s 550: learn: 0.2161712 total: 5.08s remaining: 4.14s 551: learn: 0.2160162 total: 5.09s remaining: 4.13s 552: learn: 0.2158395 total: 5.1s remaining: 4.12s 553: learn: 0.2156999 total: 5.11s remaining: 4.11s 554: learn: 0.2155332 total: 5.12s remaining: 4.1s 555: learn: 0.2153832 total: 5.13s remaining: 4.09s 556: learn: 0.2152476 total: 5.14s remaining: 4.08s 557: learn: 0.2151527 total: 5.14s remaining: 4.08s 558: learn: 0.2150890 total: 5.15s remaining: 4.07s 559: learn: 0.2149646 total: 5.16s remaining: 4.06s 560: learn: 0.2148232 total: 5.17s remaining: 4.05s 561: learn: 0.2147155 total: 5.18s remaining: 4.04s 562: learn: 0.2146310 total: 5.19s remaining: 4.03s 563: learn: 0.2144922 total: 5.2s remaining: 4.02s 564: learn: 0.2143257 total: 5.21s remaining: 4.01s 565: learn: 0.2141994 total: 5.22s remaining: 4s 566: learn: 0.2140495 total: 5.22s remaining: 3.99s 567: learn: 0.2139268 total: 5.23s remaining: 3.98s 568: learn: 0.2138160 total: 5.24s remaining: 3.97s 569: learn: 0.2136945 total: 5.25s remaining: 3.96s 570: learn: 0.2136259 total: 5.26s remaining: 3.95s 571: learn: 0.2135370 total: 5.27s remaining: 3.94s 572: learn: 0.2134327 total: 5.28s remaining: 3.93s 573: learn: 0.2133203 total: 5.29s remaining: 3.92s 574: learn: 0.2132686 total: 5.29s remaining: 3.91s 575: learn: 0.2131827 total: 5.3s remaining: 3.9s 576: learn: 0.2130532 total: 5.31s remaining: 3.89s 577: learn: 0.2129337 total: 5.32s remaining: 3.88s 578: learn: 0.2127536 total: 5.33s remaining: 3.87s 579: learn: 0.2126113 total: 5.34s remaining: 3.86s 580: learn: 0.2125353 total: 5.34s remaining: 3.85s 581: learn: 0.2124490 total: 5.35s remaining: 3.84s 582: learn: 0.2123828 total: 5.36s remaining: 3.83s 583: learn: 0.2122520 total: 5.37s remaining: 3.83s 584: learn: 0.2120871 total: 5.38s remaining: 3.82s 585: learn: 0.2119584 total: 5.39s remaining: 3.81s 586: learn: 0.2118053 total: 5.4s remaining: 3.8s 587: learn: 0.2117524 total: 5.41s remaining: 3.79s 588: learn: 0.2116625 total: 5.42s remaining: 3.78s 589: learn: 0.2115743 total: 5.42s remaining: 3.77s 590: learn: 0.2113882 total: 5.43s remaining: 3.76s 591: learn: 0.2112835 total: 5.44s remaining: 3.75s 592: learn: 0.2111451 total: 5.45s remaining: 3.74s 593: learn: 0.2110287 total: 5.46s remaining: 3.73s 594: learn: 0.2109218 total: 5.47s remaining: 3.72s 595: learn: 0.2108180 total: 5.48s remaining: 3.71s 596: learn: 0.2107286 total: 5.49s remaining: 3.71s 597: learn: 0.2104909 total: 5.5s remaining: 3.7s 598: learn: 0.2104028 total: 5.51s remaining: 3.69s 599: learn: 0.2103004 total: 5.52s remaining: 3.68s 600: learn: 0.2102437 total: 5.53s remaining: 3.67s 601: learn: 0.2101214 total: 5.54s remaining: 3.66s 602: learn: 0.2100038 total: 5.55s remaining: 3.65s 603: learn: 0.2098937 total: 5.56s remaining: 3.64s 604: learn: 0.2097939 total: 5.57s remaining: 3.63s 605: learn: 0.2097039 total: 5.58s remaining: 3.63s 606: learn: 0.2095783 total: 5.59s remaining: 3.62s 607: learn: 0.2094771 total: 5.6s remaining: 3.61s 608: learn: 0.2093728 total: 5.61s remaining: 3.6s 609: learn: 0.2092306 total: 5.62s remaining: 3.59s 610: learn: 0.2091699 total: 5.63s remaining: 3.58s 611: learn: 0.2090960 total: 5.63s remaining: 3.57s 612: learn: 0.2089333 total: 5.64s remaining: 3.56s 613: learn: 0.2087970 total: 5.65s remaining: 3.55s 614: learn: 0.2086565 total: 5.66s remaining: 3.54s 615: learn: 0.2085229 total: 5.67s remaining: 3.53s 616: learn: 0.2084539 total: 5.68s remaining: 3.52s 617: learn: 0.2083963 total: 5.69s remaining: 3.52s 618: learn: 0.2083443 total: 5.7s remaining: 3.51s 619: learn: 0.2082679 total: 5.71s remaining: 3.5s 620: learn: 0.2082152 total: 5.71s remaining: 3.49s 621: learn: 0.2080823 total: 5.73s remaining: 3.48s 622: learn: 0.2079806 total: 5.74s remaining: 3.47s 623: learn: 0.2078694 total: 5.75s remaining: 3.46s 624: learn: 0.2077670 total: 5.76s remaining: 3.45s 625: learn: 0.2076134 total: 5.77s remaining: 3.44s 626: learn: 0.2075270 total: 5.78s remaining: 3.44s 627: learn: 0.2074015 total: 5.79s remaining: 3.43s 628: learn: 0.2072918 total: 5.79s remaining: 3.42s 629: learn: 0.2071480 total: 5.8s remaining: 3.41s 630: learn: 0.2070629 total: 5.81s remaining: 3.4s 631: learn: 0.2068901 total: 5.82s remaining: 3.39s 632: learn: 0.2068071 total: 5.83s remaining: 3.38s 633: learn: 0.2066510 total: 5.84s remaining: 3.37s 634: learn: 0.2065085 total: 5.87s remaining: 3.37s 635: learn: 0.2064441 total: 5.88s remaining: 3.36s 636: learn: 0.2063597 total: 5.89s remaining: 3.35s 637: learn: 0.2062872 total: 5.9s remaining: 3.35s 638: learn: 0.2061199 total: 5.91s remaining: 3.34s 639: learn: 0.2060264 total: 5.91s remaining: 3.33s 640: learn: 0.2059275 total: 5.92s remaining: 3.32s 641: learn: 0.2058374 total: 5.93s remaining: 3.31s 642: learn: 0.2056859 total: 5.94s remaining: 3.3s 643: learn: 0.2056582 total: 5.95s remaining: 3.29s 644: learn: 0.2055508 total: 5.96s remaining: 3.28s 645: learn: 0.2054240 total: 5.97s remaining: 3.27s 646: learn: 0.2052568 total: 5.98s remaining: 3.26s 647: learn: 0.2051464 total: 5.99s remaining: 3.25s 648: learn: 0.2050658 total: 6s remaining: 3.25s 649: learn: 0.2049983 total: 6.03s remaining: 3.25s 650: learn: 0.2048593 total: 6.04s remaining: 3.24s 651: learn: 0.2047436 total: 6.05s remaining: 3.23s 652: learn: 0.2046263 total: 6.06s remaining: 3.22s 653: learn: 0.2045206 total: 6.07s remaining: 3.21s 654: learn: 0.2044644 total: 6.08s remaining: 3.2s 655: learn: 0.2043938 total: 6.08s remaining: 3.19s 656: learn: 0.2042117 total: 6.09s remaining: 3.18s 657: learn: 0.2040681 total: 6.1s remaining: 3.17s 658: learn: 0.2039429 total: 6.11s remaining: 3.16s 659: learn: 0.2038139 total: 6.12s remaining: 3.15s 660: learn: 0.2037446 total: 6.13s remaining: 3.15s 661: learn: 0.2036243 total: 6.14s remaining: 3.14s 662: learn: 0.2034967 total: 6.15s remaining: 3.13s 663: learn: 0.2034213 total: 6.16s remaining: 3.12s 664: learn: 0.2033351 total: 6.17s remaining: 3.11s 665: learn: 0.2032464 total: 6.18s remaining: 3.1s 666: learn: 0.2031178 total: 6.19s remaining: 3.09s 667: learn: 0.2030897 total: 6.2s remaining: 3.08s 668: learn: 0.2030044 total: 6.21s remaining: 3.07s 669: learn: 0.2028892 total: 6.22s remaining: 3.06s 670: learn: 0.2027634 total: 6.23s remaining: 3.05s 671: learn: 0.2026963 total: 6.24s remaining: 3.04s 672: learn: 0.2025999 total: 6.25s remaining: 3.04s 673: learn: 0.2024389 total: 6.26s remaining: 3.03s 674: learn: 0.2023066 total: 6.27s remaining: 3.02s 675: learn: 0.2022085 total: 6.28s remaining: 3.01s 676: learn: 0.2021352 total: 6.29s remaining: 3s 677: learn: 0.2020618 total: 6.29s remaining: 2.99s 678: learn: 0.2019450 total: 6.3s remaining: 2.98s 679: learn: 0.2018868 total: 6.31s remaining: 2.97s 680: learn: 0.2018401 total: 6.32s remaining: 2.96s 681: learn: 0.2017235 total: 6.33s remaining: 2.95s 682: learn: 0.2016072 total: 6.34s remaining: 2.94s 683: learn: 0.2015014 total: 6.35s remaining: 2.94s 684: learn: 0.2014332 total: 6.36s remaining: 2.93s 685: learn: 0.2013360 total: 6.37s remaining: 2.92s 686: learn: 0.2012819 total: 6.38s remaining: 2.91s 687: learn: 0.2011922 total: 6.39s remaining: 2.9s 688: learn: 0.2011158 total: 6.4s remaining: 2.89s 689: learn: 0.2009604 total: 6.41s remaining: 2.88s 690: learn: 0.2008475 total: 6.42s remaining: 2.87s 691: learn: 0.2007763 total: 6.43s remaining: 2.86s 692: learn: 0.2007123 total: 6.44s remaining: 2.85s 693: learn: 0.2005868 total: 6.45s remaining: 2.84s 694: learn: 0.2004046 total: 6.46s remaining: 2.83s 695: learn: 0.2002514 total: 6.47s remaining: 2.83s 696: learn: 0.2001676 total: 6.48s remaining: 2.82s 697: learn: 0.2001014 total: 6.49s remaining: 2.81s 698: learn: 0.2000354 total: 6.5s remaining: 2.8s 699: learn: 0.1999966 total: 6.51s remaining: 2.79s 700: learn: 0.1999178 total: 6.52s remaining: 2.78s 701: learn: 0.1998393 total: 6.53s remaining: 2.77s 702: learn: 0.1997021 total: 6.54s remaining: 2.76s 703: learn: 0.1996132 total: 6.55s remaining: 2.75s 704: learn: 0.1995008 total: 6.56s remaining: 2.75s 705: learn: 0.1994306 total: 6.57s remaining: 2.74s 706: learn: 0.1993650 total: 6.58s remaining: 2.73s 707: learn: 0.1992605 total: 6.59s remaining: 2.72s 708: learn: 0.1991203 total: 6.6s remaining: 2.71s 709: learn: 0.1990744 total: 6.61s remaining: 2.7s 710: learn: 0.1989432 total: 6.62s remaining: 2.69s 711: learn: 0.1987643 total: 6.63s remaining: 2.68s 712: learn: 0.1986177 total: 6.64s remaining: 2.67s 713: learn: 0.1985549 total: 6.65s remaining: 2.66s 714: learn: 0.1984282 total: 6.66s remaining: 2.65s 715: learn: 0.1983796 total: 6.67s remaining: 2.64s 716: learn: 0.1983275 total: 6.68s remaining: 2.64s 717: learn: 0.1982238 total: 6.7s remaining: 2.63s 718: learn: 0.1981291 total: 6.7s remaining: 2.62s 719: learn: 0.1980462 total: 6.71s remaining: 2.61s 720: learn: 0.1979671 total: 6.72s remaining: 2.6s 721: learn: 0.1978990 total: 6.73s remaining: 2.59s 722: learn: 0.1978289 total: 6.74s remaining: 2.58s 723: learn: 0.1977390 total: 6.75s remaining: 2.57s 724: learn: 0.1976629 total: 6.76s remaining: 2.56s 725: learn: 0.1975550 total: 6.77s remaining: 2.56s 726: learn: 0.1974396 total: 6.78s remaining: 2.55s 727: learn: 0.1973852 total: 6.79s remaining: 2.54s 728: learn: 0.1972859 total: 6.8s remaining: 2.53s 729: learn: 0.1972594 total: 6.81s remaining: 2.52s 730: learn: 0.1971583 total: 6.82s remaining: 2.51s 731: learn: 0.1970390 total: 6.83s remaining: 2.5s 732: learn: 0.1969296 total: 6.84s remaining: 2.49s 733: learn: 0.1968148 total: 6.85s remaining: 2.48s 734: learn: 0.1967473 total: 6.86s remaining: 2.47s 735: learn: 0.1966606 total: 6.87s remaining: 2.46s 736: learn: 0.1965283 total: 6.88s remaining: 2.45s 737: learn: 0.1963679 total: 6.88s remaining: 2.44s 738: learn: 0.1962207 total: 6.89s remaining: 2.44s 739: learn: 0.1960526 total: 6.91s remaining: 2.43s 740: learn: 0.1959885 total: 6.92s remaining: 2.42s 741: learn: 0.1959484 total: 6.92s remaining: 2.41s 742: learn: 0.1958397 total: 6.93s remaining: 2.4s 743: learn: 0.1957125 total: 6.94s remaining: 2.39s 744: learn: 0.1956444 total: 6.95s remaining: 2.38s 745: learn: 0.1955673 total: 6.96s remaining: 2.37s 746: learn: 0.1955154 total: 6.97s remaining: 2.36s 747: learn: 0.1953868 total: 6.98s remaining: 2.35s 748: learn: 0.1953089 total: 6.99s remaining: 2.34s 749: learn: 0.1951873 total: 7s remaining: 2.33s 750: learn: 0.1951182 total: 7.01s remaining: 2.33s 751: learn: 0.1950463 total: 7.02s remaining: 2.32s 752: learn: 0.1949470 total: 7.05s remaining: 2.31s 753: learn: 0.1948002 total: 7.06s remaining: 2.3s 754: learn: 0.1946782 total: 7.07s remaining: 2.29s 755: learn: 0.1946093 total: 7.08s remaining: 2.29s 756: learn: 0.1944906 total: 7.09s remaining: 2.28s 757: learn: 0.1943957 total: 7.1s remaining: 2.27s 758: learn: 0.1942928 total: 7.11s remaining: 2.26s 759: learn: 0.1942170 total: 7.12s remaining: 2.25s 760: learn: 0.1941186 total: 7.13s remaining: 2.24s 761: learn: 0.1940506 total: 7.14s remaining: 2.23s 762: learn: 0.1939629 total: 7.15s remaining: 2.22s 763: learn: 0.1938914 total: 7.16s remaining: 2.21s 764: learn: 0.1938302 total: 7.17s remaining: 2.2s 765: learn: 0.1936460 total: 7.18s remaining: 2.19s 766: learn: 0.1935484 total: 7.19s remaining: 2.18s 767: learn: 0.1934662 total: 7.2s remaining: 2.17s 768: learn: 0.1933517 total: 7.21s remaining: 2.16s 769: learn: 0.1932679 total: 7.21s remaining: 2.15s 770: learn: 0.1932146 total: 7.22s remaining: 2.15s 771: learn: 0.1931580 total: 7.23s remaining: 2.14s 772: learn: 0.1930608 total: 7.24s remaining: 2.13s 773: learn: 0.1929738 total: 7.25s remaining: 2.12s 774: learn: 0.1928951 total: 7.26s remaining: 2.11s 775: learn: 0.1928149 total: 7.27s remaining: 2.1s 776: learn: 0.1927089 total: 7.29s remaining: 2.09s 777: learn: 0.1925211 total: 7.3s remaining: 2.08s 778: learn: 0.1923738 total: 7.31s remaining: 2.07s 779: learn: 0.1923381 total: 7.32s remaining: 2.06s 780: learn: 0.1921897 total: 7.33s remaining: 2.06s 781: learn: 0.1920827 total: 7.34s remaining: 2.05s 782: learn: 0.1919364 total: 7.35s remaining: 2.04s 783: learn: 0.1917910 total: 7.36s remaining: 2.03s 784: learn: 0.1916897 total: 7.37s remaining: 2.02s 785: learn: 0.1916133 total: 7.38s remaining: 2.01s 786: learn: 0.1914993 total: 7.39s remaining: 2s 787: learn: 0.1913764 total: 7.4s remaining: 1.99s 788: learn: 0.1912800 total: 7.41s remaining: 1.98s 789: learn: 0.1911654 total: 7.42s remaining: 1.97s 790: learn: 0.1911265 total: 7.43s remaining: 1.96s 791: learn: 0.1910414 total: 7.44s remaining: 1.95s 792: learn: 0.1909511 total: 7.45s remaining: 1.94s 793: learn: 0.1908665 total: 7.46s remaining: 1.93s 794: learn: 0.1907065 total: 7.47s remaining: 1.93s 795: learn: 0.1906451 total: 7.48s remaining: 1.92s 796: learn: 0.1905599 total: 7.49s remaining: 1.91s 797: learn: 0.1904331 total: 7.5s remaining: 1.9s 798: learn: 0.1903862 total: 7.51s remaining: 1.89s 799: learn: 0.1902897 total: 7.52s remaining: 1.88s 800: learn: 0.1902358 total: 7.53s remaining: 1.87s 801: learn: 0.1901234 total: 7.53s remaining: 1.86s 802: learn: 0.1900622 total: 7.54s remaining: 1.85s 803: learn: 0.1899813 total: 7.55s remaining: 1.84s 804: learn: 0.1898638 total: 7.56s remaining: 1.83s 805: learn: 0.1897952 total: 7.57s remaining: 1.82s 806: learn: 0.1897101 total: 7.58s remaining: 1.81s 807: learn: 0.1896124 total: 7.59s remaining: 1.8s 808: learn: 0.1895211 total: 7.59s remaining: 1.79s 809: learn: 0.1894348 total: 7.6s remaining: 1.78s 810: learn: 0.1893045 total: 7.61s remaining: 1.77s 811: learn: 0.1891806 total: 7.62s remaining: 1.76s 812: learn: 0.1890739 total: 7.63s remaining: 1.75s 813: learn: 0.1889652 total: 7.64s remaining: 1.75s 814: learn: 0.1888606 total: 7.65s remaining: 1.74s 815: learn: 0.1887620 total: 7.66s remaining: 1.73s 816: learn: 0.1887163 total: 7.67s remaining: 1.72s 817: learn: 0.1886679 total: 7.68s remaining: 1.71s 818: learn: 0.1885343 total: 7.68s remaining: 1.7s 819: learn: 0.1884173 total: 7.69s remaining: 1.69s 820: learn: 0.1883315 total: 7.7s remaining: 1.68s 821: learn: 0.1882177 total: 7.71s remaining: 1.67s 822: learn: 0.1880899 total: 7.72s remaining: 1.66s 823: learn: 0.1879901 total: 7.73s remaining: 1.65s 824: learn: 0.1878798 total: 7.74s remaining: 1.64s 825: learn: 0.1877830 total: 7.75s remaining: 1.63s 826: learn: 0.1876515 total: 7.75s remaining: 1.62s 827: learn: 0.1875839 total: 7.76s remaining: 1.61s 828: learn: 0.1874984 total: 7.77s remaining: 1.6s 829: learn: 0.1873354 total: 7.78s remaining: 1.59s 830: learn: 0.1871765 total: 7.79s remaining: 1.58s 831: learn: 0.1870846 total: 7.8s remaining: 1.57s 832: learn: 0.1869671 total: 7.81s remaining: 1.56s 833: learn: 0.1868693 total: 7.82s remaining: 1.55s 834: learn: 0.1867897 total: 7.82s remaining: 1.55s 835: learn: 0.1867156 total: 7.83s remaining: 1.54s 836: learn: 0.1866729 total: 7.84s remaining: 1.53s 837: learn: 0.1865581 total: 7.85s remaining: 1.52s 838: learn: 0.1864680 total: 7.86s remaining: 1.51s 839: learn: 0.1863957 total: 7.87s remaining: 1.5s 840: learn: 0.1863220 total: 7.88s remaining: 1.49s 841: learn: 0.1862263 total: 7.89s remaining: 1.48s 842: learn: 0.1861147 total: 7.89s remaining: 1.47s 843: learn: 0.1860306 total: 7.9s remaining: 1.46s 844: learn: 0.1859270 total: 7.91s remaining: 1.45s 845: learn: 0.1857666 total: 7.92s remaining: 1.44s 846: learn: 0.1856550 total: 7.93s remaining: 1.43s 847: learn: 0.1855864 total: 7.94s remaining: 1.42s 848: learn: 0.1854999 total: 7.95s remaining: 1.41s 849: learn: 0.1853941 total: 7.95s remaining: 1.4s 850: learn: 0.1853014 total: 7.96s remaining: 1.39s 851: learn: 0.1851548 total: 7.97s remaining: 1.38s 852: learn: 0.1850907 total: 7.98s remaining: 1.38s 853: learn: 0.1850042 total: 7.99s remaining: 1.36s 854: learn: 0.1848861 total: 8s remaining: 1.36s 855: learn: 0.1848646 total: 8.01s remaining: 1.35s 856: learn: 0.1848044 total: 8.01s remaining: 1.34s 857: learn: 0.1847133 total: 8.02s remaining: 1.33s 858: learn: 0.1846143 total: 8.03s remaining: 1.32s 859: learn: 0.1845560 total: 8.04s remaining: 1.31s 860: learn: 0.1844811 total: 8.05s remaining: 1.3s 861: learn: 0.1843629 total: 8.09s remaining: 1.29s 862: learn: 0.1842635 total: 8.09s remaining: 1.28s 863: learn: 0.1841471 total: 8.1s remaining: 1.27s 864: learn: 0.1840235 total: 8.11s remaining: 1.27s 865: learn: 0.1839029 total: 8.12s remaining: 1.26s 866: learn: 0.1837678 total: 8.13s remaining: 1.25s 867: learn: 0.1836684 total: 8.14s remaining: 1.24s 868: learn: 0.1836005 total: 8.15s remaining: 1.23s 869: learn: 0.1835241 total: 8.16s remaining: 1.22s 870: learn: 0.1834242 total: 8.16s remaining: 1.21s 871: learn: 0.1833506 total: 8.17s remaining: 1.2s 872: learn: 0.1832906 total: 8.18s remaining: 1.19s 873: learn: 0.1831817 total: 8.19s remaining: 1.18s 874: learn: 0.1830536 total: 8.2s remaining: 1.17s 875: learn: 0.1828993 total: 8.21s remaining: 1.16s 876: learn: 0.1828154 total: 8.22s remaining: 1.15s 877: learn: 0.1827272 total: 8.23s remaining: 1.14s 878: learn: 0.1826231 total: 8.24s remaining: 1.13s 879: learn: 0.1825376 total: 8.24s remaining: 1.12s 880: learn: 0.1824464 total: 8.25s remaining: 1.11s 881: learn: 0.1823522 total: 8.26s remaining: 1.1s 882: learn: 0.1822651 total: 8.27s remaining: 1.1s 883: learn: 0.1821848 total: 8.28s remaining: 1.09s 884: learn: 0.1821301 total: 8.29s remaining: 1.08s 885: learn: 0.1820125 total: 8.3s remaining: 1.07s 886: learn: 0.1819399 total: 8.31s remaining: 1.06s 887: learn: 0.1818140 total: 8.31s remaining: 1.05s 888: learn: 0.1817140 total: 8.32s remaining: 1.04s 889: learn: 0.1815989 total: 8.33s remaining: 1.03s 890: learn: 0.1815156 total: 8.34s remaining: 1.02s 891: learn: 0.1814488 total: 8.35s remaining: 1.01s 892: learn: 0.1813638 total: 8.36s remaining: 1s 893: learn: 0.1813008 total: 8.37s remaining: 992ms 894: learn: 0.1812216 total: 8.38s remaining: 983ms 895: learn: 0.1812044 total: 8.38s remaining: 973ms 896: learn: 0.1811010 total: 8.39s remaining: 964ms 897: learn: 0.1809685 total: 8.4s remaining: 954ms 898: learn: 0.1809030 total: 8.41s remaining: 945ms 899: learn: 0.1808305 total: 8.42s remaining: 936ms 900: learn: 0.1807861 total: 8.43s remaining: 926ms 901: learn: 0.1806653 total: 8.44s remaining: 917ms 902: learn: 0.1805296 total: 8.45s remaining: 907ms 903: learn: 0.1804508 total: 8.46s remaining: 898ms 904: learn: 0.1803622 total: 8.46s remaining: 888ms 905: learn: 0.1802344 total: 8.47s remaining: 879ms 906: learn: 0.1801596 total: 8.48s remaining: 870ms 907: learn: 0.1800688 total: 8.49s remaining: 860ms 908: learn: 0.1799939 total: 8.5s remaining: 851ms 909: learn: 0.1799342 total: 8.51s remaining: 842ms 910: learn: 0.1798473 total: 8.52s remaining: 832ms 911: learn: 0.1797966 total: 8.53s remaining: 823ms 912: learn: 0.1796393 total: 8.54s remaining: 814ms 913: learn: 0.1795535 total: 8.55s remaining: 804ms 914: learn: 0.1794407 total: 8.56s remaining: 795ms 915: learn: 0.1793746 total: 8.57s remaining: 786ms 916: learn: 0.1793250 total: 8.57s remaining: 776ms 917: learn: 0.1792603 total: 8.59s remaining: 767ms 918: learn: 0.1791540 total: 8.59s remaining: 758ms 919: learn: 0.1790917 total: 8.6s remaining: 748ms 920: learn: 0.1790189 total: 8.61s remaining: 739ms 921: learn: 0.1789184 total: 8.62s remaining: 729ms 922: learn: 0.1788156 total: 8.63s remaining: 720ms 923: learn: 0.1787869 total: 8.64s remaining: 710ms 924: learn: 0.1786868 total: 8.65s remaining: 701ms 925: learn: 0.1785820 total: 8.66s remaining: 692ms 926: learn: 0.1784979 total: 8.66s remaining: 682ms 927: learn: 0.1783744 total: 8.68s remaining: 673ms 928: learn: 0.1783027 total: 8.68s remaining: 664ms 929: learn: 0.1782228 total: 8.69s remaining: 654ms 930: learn: 0.1781661 total: 8.7s remaining: 645ms 931: learn: 0.1781153 total: 8.71s remaining: 636ms 932: learn: 0.1780070 total: 8.72s remaining: 626ms 933: learn: 0.1779546 total: 8.73s remaining: 617ms 934: learn: 0.1778968 total: 8.74s remaining: 607ms 935: learn: 0.1778073 total: 8.74s remaining: 598ms 936: learn: 0.1777226 total: 8.75s remaining: 589ms 937: learn: 0.1776164 total: 8.76s remaining: 579ms 938: learn: 0.1775034 total: 8.77s remaining: 570ms 939: learn: 0.1774314 total: 8.78s remaining: 561ms 940: learn: 0.1773821 total: 8.79s remaining: 551ms 941: learn: 0.1772697 total: 8.8s remaining: 542ms 942: learn: 0.1772028 total: 8.81s remaining: 532ms 943: learn: 0.1770200 total: 8.82s remaining: 523ms 944: learn: 0.1769435 total: 8.82s remaining: 514ms 945: learn: 0.1768410 total: 8.83s remaining: 504ms 946: learn: 0.1767668 total: 8.84s remaining: 495ms 947: learn: 0.1766897 total: 8.85s remaining: 485ms 948: learn: 0.1765567 total: 8.86s remaining: 476ms 949: learn: 0.1764947 total: 8.87s remaining: 467ms 950: learn: 0.1764343 total: 8.88s remaining: 457ms 951: learn: 0.1763480 total: 8.88s remaining: 448ms 952: learn: 0.1762532 total: 8.89s remaining: 439ms 953: learn: 0.1761541 total: 8.9s remaining: 429ms 954: learn: 0.1760157 total: 8.91s remaining: 420ms 955: learn: 0.1759635 total: 8.92s remaining: 410ms 956: learn: 0.1758474 total: 8.93s remaining: 401ms 957: learn: 0.1757974 total: 8.94s remaining: 392ms 958: learn: 0.1757532 total: 8.94s remaining: 382ms 959: learn: 0.1756419 total: 8.95s remaining: 373ms 960: learn: 0.1756037 total: 8.96s remaining: 364ms 961: learn: 0.1755298 total: 8.97s remaining: 354ms 962: learn: 0.1754178 total: 8.98s remaining: 345ms 963: learn: 0.1753233 total: 8.99s remaining: 336ms 964: learn: 0.1752627 total: 9s remaining: 326ms 965: learn: 0.1751925 total: 9.01s remaining: 317ms 966: learn: 0.1750914 total: 9.01s remaining: 308ms 967: learn: 0.1750259 total: 9.02s remaining: 298ms 968: learn: 0.1749447 total: 9.03s remaining: 289ms 969: learn: 0.1748836 total: 9.04s remaining: 280ms 970: learn: 0.1748347 total: 9.05s remaining: 270ms 971: learn: 0.1747652 total: 9.06s remaining: 261ms 972: learn: 0.1747124 total: 9.07s remaining: 252ms 973: learn: 0.1746313 total: 9.1s remaining: 243ms 974: learn: 0.1746143 total: 9.11s remaining: 234ms 975: learn: 0.1745627 total: 9.12s remaining: 224ms 976: learn: 0.1745104 total: 9.13s remaining: 215ms 977: learn: 0.1744543 total: 9.13s remaining: 205ms 978: learn: 0.1743813 total: 9.14s remaining: 196ms 979: learn: 0.1743098 total: 9.15s remaining: 187ms 980: learn: 0.1742651 total: 9.16s remaining: 177ms 981: learn: 0.1742278 total: 9.17s remaining: 168ms 982: learn: 0.1741871 total: 9.18s remaining: 159ms 983: learn: 0.1740969 total: 9.19s remaining: 149ms 984: learn: 0.1739942 total: 9.2s remaining: 140ms 985: learn: 0.1739048 total: 9.2s remaining: 131ms 986: learn: 0.1738052 total: 9.21s remaining: 121ms 987: learn: 0.1737144 total: 9.22s remaining: 112ms 988: learn: 0.1735685 total: 9.23s remaining: 103ms 989: learn: 0.1734131 total: 9.24s remaining: 93.3ms 990: learn: 0.1733206 total: 9.25s remaining: 84ms 991: learn: 0.1732759 total: 9.26s remaining: 74.6ms 992: learn: 0.1732386 total: 9.26s remaining: 65.3ms 993: learn: 0.1731644 total: 9.27s remaining: 56ms 994: learn: 0.1730383 total: 9.28s remaining: 46.6ms 995: learn: 0.1729884 total: 9.29s remaining: 37.3ms 996: learn: 0.1729129 total: 9.3s remaining: 28ms 997: learn: 0.1728584 total: 9.31s remaining: 18.6ms 998: learn: 0.1727712 total: 9.31s remaining: 9.32ms 999: learn: 0.1727183 total: 9.32s remaining: 0us
<catboost.core.CatBoostClassifier at 0x228e48eec40>
# Feature Importances : the importance of the feauture that contributed to the learning of our model sroted from most important to least important
def feature_importance(model, data):
"""
Function to show which features are most important in the model.
::param_model:: Which model to use?
::param_data:: What data to use?
"""
fea_imp = pd.DataFrame({'imp': model.feature_importances_, 'col': data.columns})
fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]
_= fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))
return fea_imp
feature_importance(catboost_model, X_train)
| imp | col | |
|---|---|---|
| 11 | 1.213266 | forecast_price_pow_off_peak |
| 3 | 1.254589 | cons_gas_12m |
| 6 | 1.352416 | forecast_cons_year |
| 13 | 1.576614 | imp_cons |
| 10 | 1.934195 | forecast_price_energy_peak |
| 1 | 1.992084 | channel_sales |
| 23 | 2.070375 | year_date_activ |
| 18 | 2.143501 | num_years_antig |
| 31 | 2.187367 | day_date_modif_prod |
| 30 | 2.310598 | month_date_modif_prod |
| 24 | 2.418985 | month_date_activ |
| 19 | 2.483177 | origin_up |
| 33 | 2.513828 | month_date_renewal |
| 34 | 2.528226 | day_date_renewal |
| 29 | 2.547458 | year_date_modif_prod |
| 28 | 2.548470 | day_date_end |
| 25 | 2.916687 | day_date_activ |
| 22 | 3.074912 | fore_casted_price_energy_total |
| 9 | 3.084097 | forecast_price_energy_off_peak |
| 27 | 3.674631 | month_date_end |
| 5 | 4.124847 | forecast_cons_12m |
| 2 | 4.349512 | cons_12m |
| 0 | 4.481502 | id |
| 17 | 4.542184 | net_margin |
| 21 | 4.784779 | total_cons_12m |
| 4 | 5.258148 | cons_last_month |
| 8 | 5.543921 | forecast_meter_rent_12m |
| 15 | 5.792107 | margin_net_pow_ele |
| 14 | 5.925329 | margin_gross_pow_ele |
| 20 | 7.133139 | pow_max |
As we can see for this model we can clearly notice that we have four features with importances under 1 comapred to most of the features ranging between 3 and 7.5
# X_train.drop(columns=["has_gas","nb_prod_act","num_years_antig","forecast_discount_energy"], inplace=True)
# X_test.drop(columns=["has_gas","nb_prod_act","num_years_antig","forecast_discount_energy"], inplace=True)
Let's begin by training our Catboost model and testing it on the test dataset and see how well it performs
train_pool = Pool(X_train,y_train)
# CatBoost model definition
catboost_model = CatBoostClassifier(
iterations=1000,
custom_loss=['Accuracy'],
loss_function='CrossEntropy',depth=5
)
# Fit CatBoost model
catboost_model.fit(train_pool,
plot=True)
# CatBoost accuracy
acc_catboost = round(catboost_model.score(X_train, y_train) * 100, 2)
MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))
0: learn: 0.6659824 total: 25.8ms remaining: 25.8s 1: learn: 0.6405579 total: 34.3ms remaining: 17.1s 2: learn: 0.6175952 total: 42.9ms remaining: 14.3s 3: learn: 0.5957692 total: 50.9ms remaining: 12.7s 4: learn: 0.5753977 total: 58.8ms remaining: 11.7s 5: learn: 0.5566162 total: 66.6ms remaining: 11s 6: learn: 0.5390072 total: 74.3ms remaining: 10.5s 7: learn: 0.5232824 total: 81.9ms remaining: 10.2s 8: learn: 0.5078731 total: 89.8ms remaining: 9.88s 9: learn: 0.4942584 total: 97.5ms remaining: 9.65s 10: learn: 0.4817198 total: 105ms remaining: 9.45s 11: learn: 0.4701478 total: 113ms remaining: 9.29s 12: learn: 0.4589232 total: 120ms remaining: 9.14s 13: learn: 0.4487878 total: 128ms remaining: 9.02s 14: learn: 0.4394818 total: 136ms remaining: 8.92s 15: learn: 0.4297546 total: 144ms remaining: 8.83s 16: learn: 0.4214325 total: 151ms remaining: 8.74s 17: learn: 0.4141087 total: 159ms remaining: 8.66s 18: learn: 0.4072672 total: 167ms remaining: 8.6s 19: learn: 0.4005293 total: 174ms remaining: 8.55s 20: learn: 0.3946043 total: 182ms remaining: 8.49s 21: learn: 0.3885827 total: 190ms remaining: 8.44s 22: learn: 0.3832021 total: 198ms remaining: 8.4s 23: learn: 0.3783827 total: 205ms remaining: 8.35s 24: learn: 0.3735162 total: 213ms remaining: 8.31s 25: learn: 0.3693656 total: 222ms remaining: 8.3s 26: learn: 0.3654390 total: 230ms remaining: 8.29s 27: learn: 0.3615553 total: 238ms remaining: 8.27s 28: learn: 0.3580022 total: 246ms remaining: 8.25s 29: learn: 0.3543967 total: 254ms remaining: 8.23s 30: learn: 0.3511688 total: 262ms remaining: 8.2s 31: learn: 0.3480313 total: 270ms remaining: 8.17s 32: learn: 0.3455761 total: 278ms remaining: 8.14s 33: learn: 0.3433082 total: 286ms remaining: 8.11s 34: learn: 0.3405098 total: 294ms remaining: 8.1s 35: learn: 0.3382171 total: 302ms remaining: 8.08s 36: learn: 0.3362031 total: 309ms remaining: 8.05s 37: learn: 0.3341666 total: 317ms remaining: 8.02s 38: learn: 0.3324831 total: 324ms remaining: 7.99s 39: learn: 0.3308527 total: 332ms remaining: 7.97s 40: learn: 0.3292917 total: 340ms remaining: 7.95s 41: learn: 0.3278054 total: 347ms remaining: 7.92s 42: learn: 0.3261658 total: 355ms remaining: 7.9s 43: learn: 0.3248788 total: 363ms remaining: 7.88s 44: learn: 0.3233453 total: 370ms remaining: 7.86s 45: learn: 0.3222141 total: 378ms remaining: 7.84s 46: learn: 0.3208569 total: 385ms remaining: 7.81s 47: learn: 0.3196304 total: 393ms remaining: 7.8s 48: learn: 0.3186704 total: 401ms remaining: 7.78s 49: learn: 0.3174357 total: 408ms remaining: 7.76s 50: learn: 0.3164986 total: 417ms remaining: 7.75s 51: learn: 0.3154912 total: 425ms remaining: 7.75s 52: learn: 0.3146420 total: 433ms remaining: 7.74s 53: learn: 0.3139497 total: 441ms remaining: 7.72s 54: learn: 0.3133423 total: 449ms remaining: 7.71s 55: learn: 0.3125126 total: 458ms remaining: 7.71s 56: learn: 0.3119221 total: 466ms remaining: 7.71s 57: learn: 0.3111901 total: 474ms remaining: 7.7s 58: learn: 0.3106373 total: 482ms remaining: 7.68s 59: learn: 0.3099091 total: 490ms remaining: 7.67s 60: learn: 0.3092748 total: 498ms remaining: 7.66s 61: learn: 0.3086159 total: 506ms remaining: 7.65s 62: learn: 0.3081743 total: 513ms remaining: 7.63s 63: learn: 0.3073849 total: 521ms remaining: 7.62s 64: learn: 0.3069738 total: 529ms remaining: 7.61s 65: learn: 0.3066172 total: 537ms remaining: 7.6s 66: learn: 0.3061270 total: 545ms remaining: 7.59s 67: learn: 0.3056199 total: 553ms remaining: 7.57s 68: learn: 0.3051018 total: 560ms remaining: 7.56s 69: learn: 0.3044529 total: 568ms remaining: 7.54s 70: learn: 0.3040192 total: 575ms remaining: 7.53s 71: learn: 0.3034136 total: 583ms remaining: 7.52s 72: learn: 0.3030527 total: 591ms remaining: 7.5s 73: learn: 0.3027174 total: 599ms remaining: 7.5s 74: learn: 0.3024280 total: 608ms remaining: 7.5s 75: learn: 0.3020214 total: 616ms remaining: 7.49s 76: learn: 0.3016066 total: 624ms remaining: 7.48s 77: learn: 0.3013100 total: 632ms remaining: 7.47s 78: learn: 0.3008993 total: 640ms remaining: 7.46s 79: learn: 0.3004685 total: 648ms remaining: 7.45s 80: learn: 0.3002172 total: 656ms remaining: 7.44s 81: learn: 0.2999444 total: 664ms remaining: 7.43s 82: learn: 0.2995666 total: 671ms remaining: 7.42s 83: learn: 0.2993016 total: 679ms remaining: 7.41s 84: learn: 0.2988826 total: 687ms remaining: 7.39s 85: learn: 0.2986351 total: 695ms remaining: 7.38s 86: learn: 0.2983863 total: 702ms remaining: 7.37s 87: learn: 0.2980171 total: 710ms remaining: 7.36s 88: learn: 0.2977403 total: 718ms remaining: 7.35s 89: learn: 0.2974953 total: 726ms remaining: 7.34s 90: learn: 0.2973102 total: 733ms remaining: 7.32s 91: learn: 0.2969956 total: 741ms remaining: 7.31s 92: learn: 0.2967368 total: 749ms remaining: 7.3s 93: learn: 0.2965637 total: 756ms remaining: 7.29s 94: learn: 0.2963405 total: 764ms remaining: 7.28s 95: learn: 0.2961521 total: 772ms remaining: 7.27s 96: learn: 0.2958792 total: 779ms remaining: 7.25s 97: learn: 0.2956809 total: 788ms remaining: 7.25s 98: learn: 0.2954239 total: 796ms remaining: 7.25s 99: learn: 0.2951342 total: 805ms remaining: 7.24s 100: learn: 0.2948867 total: 813ms remaining: 7.24s 101: learn: 0.2946644 total: 821ms remaining: 7.23s 102: learn: 0.2942808 total: 829ms remaining: 7.22s 103: learn: 0.2940046 total: 838ms remaining: 7.22s 104: learn: 0.2938677 total: 846ms remaining: 7.21s 105: learn: 0.2935676 total: 853ms remaining: 7.2s 106: learn: 0.2931513 total: 861ms remaining: 7.19s 107: learn: 0.2929200 total: 869ms remaining: 7.18s 108: learn: 0.2926363 total: 877ms remaining: 7.17s 109: learn: 0.2925167 total: 885ms remaining: 7.16s 110: learn: 0.2922955 total: 893ms remaining: 7.15s 111: learn: 0.2921320 total: 900ms remaining: 7.14s 112: learn: 0.2919454 total: 908ms remaining: 7.13s 113: learn: 0.2918265 total: 916ms remaining: 7.12s 114: learn: 0.2915485 total: 923ms remaining: 7.11s 115: learn: 0.2913738 total: 931ms remaining: 7.09s 116: learn: 0.2912518 total: 939ms remaining: 7.09s 117: learn: 0.2911091 total: 947ms remaining: 7.08s 118: learn: 0.2908786 total: 954ms remaining: 7.07s 119: learn: 0.2906676 total: 962ms remaining: 7.05s 120: learn: 0.2905568 total: 970ms remaining: 7.04s 121: learn: 0.2903138 total: 978ms remaining: 7.04s 122: learn: 0.2901788 total: 993ms remaining: 7.08s 123: learn: 0.2900386 total: 1s remaining: 7.08s 124: learn: 0.2899042 total: 1.01s remaining: 7.08s 125: learn: 0.2897820 total: 1.02s remaining: 7.07s 126: learn: 0.2895897 total: 1.03s remaining: 7.07s 127: learn: 0.2894179 total: 1.04s remaining: 7.06s 128: learn: 0.2891133 total: 1.04s remaining: 7.06s 129: learn: 0.2889714 total: 1.05s remaining: 7.05s 130: learn: 0.2888294 total: 1.06s remaining: 7.05s 131: learn: 0.2885993 total: 1.07s remaining: 7.04s 132: learn: 0.2884202 total: 1.08s remaining: 7.04s 133: learn: 0.2882644 total: 1.09s remaining: 7.03s 134: learn: 0.2880881 total: 1.1s remaining: 7.03s 135: learn: 0.2879370 total: 1.1s remaining: 7.02s 136: learn: 0.2877882 total: 1.11s remaining: 7.01s 137: learn: 0.2875292 total: 1.12s remaining: 7.01s 138: learn: 0.2874122 total: 1.13s remaining: 7s 139: learn: 0.2872162 total: 1.14s remaining: 6.99s 140: learn: 0.2868727 total: 1.15s remaining: 6.98s 141: learn: 0.2866856 total: 1.15s remaining: 6.97s 142: learn: 0.2864487 total: 1.16s remaining: 6.96s 143: learn: 0.2862735 total: 1.17s remaining: 6.95s 144: learn: 0.2861235 total: 1.18s remaining: 6.94s 145: learn: 0.2858975 total: 1.19s remaining: 6.93s 146: learn: 0.2857673 total: 1.19s remaining: 6.93s 147: learn: 0.2855723 total: 1.2s remaining: 6.92s 148: learn: 0.2854244 total: 1.22s remaining: 6.94s 149: learn: 0.2853219 total: 1.22s remaining: 6.94s 150: learn: 0.2852068 total: 1.23s remaining: 6.93s 151: learn: 0.2850253 total: 1.24s remaining: 6.93s 152: learn: 0.2848283 total: 1.25s remaining: 6.92s 153: learn: 0.2847084 total: 1.26s remaining: 6.91s 154: learn: 0.2845471 total: 1.27s remaining: 6.9s 155: learn: 0.2843973 total: 1.27s remaining: 6.89s 156: learn: 0.2842693 total: 1.28s remaining: 6.88s 157: learn: 0.2841483 total: 1.29s remaining: 6.88s 158: learn: 0.2839903 total: 1.3s remaining: 6.87s 159: learn: 0.2838014 total: 1.31s remaining: 6.87s 160: learn: 0.2836920 total: 1.31s remaining: 6.86s 161: learn: 0.2835902 total: 1.32s remaining: 6.85s 162: learn: 0.2834765 total: 1.33s remaining: 6.84s 163: learn: 0.2833687 total: 1.34s remaining: 6.83s 164: learn: 0.2831634 total: 1.35s remaining: 6.82s 165: learn: 0.2830367 total: 1.36s remaining: 6.82s 166: learn: 0.2829258 total: 1.36s remaining: 6.81s 167: learn: 0.2827520 total: 1.37s remaining: 6.8s 168: learn: 0.2826376 total: 1.38s remaining: 6.79s 169: learn: 0.2825152 total: 1.39s remaining: 6.79s 170: learn: 0.2823954 total: 1.4s remaining: 6.78s 171: learn: 0.2822768 total: 1.41s remaining: 6.76s 172: learn: 0.2820453 total: 1.41s remaining: 6.76s 173: learn: 0.2819167 total: 1.42s remaining: 6.75s 174: learn: 0.2817064 total: 1.43s remaining: 6.74s 175: learn: 0.2815446 total: 1.44s remaining: 6.74s 176: learn: 0.2813904 total: 1.45s remaining: 6.73s 177: learn: 0.2812088 total: 1.46s remaining: 6.73s 178: learn: 0.2810681 total: 1.47s remaining: 6.72s 179: learn: 0.2806865 total: 1.47s remaining: 6.71s 180: learn: 0.2805176 total: 1.48s remaining: 6.71s 181: learn: 0.2804030 total: 1.49s remaining: 6.7s 182: learn: 0.2802564 total: 1.5s remaining: 6.69s 183: learn: 0.2800065 total: 1.51s remaining: 6.68s 184: learn: 0.2798893 total: 1.51s remaining: 6.67s 185: learn: 0.2797287 total: 1.52s remaining: 6.66s 186: learn: 0.2795906 total: 1.53s remaining: 6.65s 187: learn: 0.2794678 total: 1.54s remaining: 6.64s 188: learn: 0.2793049 total: 1.55s remaining: 6.64s 189: learn: 0.2791951 total: 1.55s remaining: 6.63s 190: learn: 0.2790785 total: 1.56s remaining: 6.62s 191: learn: 0.2789654 total: 1.57s remaining: 6.62s 192: learn: 0.2789293 total: 1.58s remaining: 6.61s 193: learn: 0.2788345 total: 1.59s remaining: 6.6s 194: learn: 0.2786405 total: 1.6s remaining: 6.59s 195: learn: 0.2784205 total: 1.6s remaining: 6.58s 196: learn: 0.2783140 total: 1.61s remaining: 6.58s 197: learn: 0.2781797 total: 1.62s remaining: 6.57s 198: learn: 0.2780585 total: 1.63s remaining: 6.56s 199: learn: 0.2779278 total: 1.64s remaining: 6.55s 200: learn: 0.2778252 total: 1.65s remaining: 6.54s 201: learn: 0.2777250 total: 1.65s remaining: 6.53s 202: learn: 0.2776351 total: 1.66s remaining: 6.53s 203: learn: 0.2774977 total: 1.67s remaining: 6.52s 204: learn: 0.2773676 total: 1.68s remaining: 6.51s 205: learn: 0.2772591 total: 1.69s remaining: 6.5s 206: learn: 0.2771230 total: 1.69s remaining: 6.49s 207: learn: 0.2769973 total: 1.7s remaining: 6.49s 208: learn: 0.2768886 total: 1.71s remaining: 6.48s 209: learn: 0.2767459 total: 1.72s remaining: 6.47s 210: learn: 0.2766523 total: 1.73s remaining: 6.46s 211: learn: 0.2765695 total: 1.74s remaining: 6.45s 212: learn: 0.2763649 total: 1.75s remaining: 6.45s 213: learn: 0.2762245 total: 1.75s remaining: 6.44s 214: learn: 0.2761119 total: 1.76s remaining: 6.43s 215: learn: 0.2760007 total: 1.77s remaining: 6.42s 216: learn: 0.2759013 total: 1.78s remaining: 6.41s 217: learn: 0.2757549 total: 1.78s remaining: 6.4s 218: learn: 0.2756298 total: 1.79s remaining: 6.39s 219: learn: 0.2754923 total: 1.8s remaining: 6.38s 220: learn: 0.2753516 total: 1.81s remaining: 6.37s 221: learn: 0.2752650 total: 1.81s remaining: 6.36s 222: learn: 0.2751821 total: 1.82s remaining: 6.35s 223: learn: 0.2750776 total: 1.83s remaining: 6.34s 224: learn: 0.2749463 total: 1.84s remaining: 6.33s 225: learn: 0.2748100 total: 1.85s remaining: 6.32s 226: learn: 0.2746633 total: 1.85s remaining: 6.31s 227: learn: 0.2745782 total: 1.86s remaining: 6.3s 228: learn: 0.2744863 total: 1.87s remaining: 6.29s 229: learn: 0.2744245 total: 1.88s remaining: 6.28s 230: learn: 0.2742987 total: 1.88s remaining: 6.27s 231: learn: 0.2741001 total: 1.89s remaining: 6.26s 232: learn: 0.2739773 total: 1.9s remaining: 6.25s 233: learn: 0.2738003 total: 1.91s remaining: 6.25s 234: learn: 0.2736854 total: 1.92s remaining: 6.24s 235: learn: 0.2733932 total: 1.93s remaining: 6.24s 236: learn: 0.2732870 total: 1.93s remaining: 6.23s 237: learn: 0.2730978 total: 1.94s remaining: 6.22s 238: learn: 0.2729357 total: 1.95s remaining: 6.21s 239: learn: 0.2728556 total: 1.96s remaining: 6.2s 240: learn: 0.2726577 total: 1.97s remaining: 6.19s 241: learn: 0.2725177 total: 1.97s remaining: 6.18s 242: learn: 0.2724568 total: 1.98s remaining: 6.17s 243: learn: 0.2723783 total: 2s remaining: 6.19s 244: learn: 0.2722647 total: 2s remaining: 6.18s 245: learn: 0.2721266 total: 2.01s remaining: 6.17s 246: learn: 0.2719629 total: 2.02s remaining: 6.16s 247: learn: 0.2718415 total: 2.03s remaining: 6.15s 248: learn: 0.2717672 total: 2.04s remaining: 6.14s 249: learn: 0.2716562 total: 2.04s remaining: 6.13s 250: learn: 0.2715273 total: 2.05s remaining: 6.12s 251: learn: 0.2713233 total: 2.06s remaining: 6.11s 252: learn: 0.2711585 total: 2.07s remaining: 6.1s 253: learn: 0.2710264 total: 2.07s remaining: 6.09s 254: learn: 0.2709238 total: 2.08s remaining: 6.08s 255: learn: 0.2707380 total: 2.09s remaining: 6.07s 256: learn: 0.2706147 total: 2.1s remaining: 6.07s 257: learn: 0.2705446 total: 2.11s remaining: 6.06s 258: learn: 0.2704140 total: 2.12s remaining: 6.05s 259: learn: 0.2702277 total: 2.12s remaining: 6.04s 260: learn: 0.2700105 total: 2.13s remaining: 6.04s 261: learn: 0.2698879 total: 2.14s remaining: 6.03s 262: learn: 0.2698188 total: 2.15s remaining: 6.02s 263: learn: 0.2697322 total: 2.15s remaining: 6.01s 264: learn: 0.2695722 total: 2.16s remaining: 6s 265: learn: 0.2694767 total: 2.17s remaining: 5.99s 266: learn: 0.2693831 total: 2.18s remaining: 5.98s 267: learn: 0.2691713 total: 2.19s remaining: 5.97s 268: learn: 0.2690294 total: 2.19s remaining: 5.96s 269: learn: 0.2689518 total: 2.2s remaining: 5.95s 270: learn: 0.2687903 total: 2.21s remaining: 5.94s 271: learn: 0.2686981 total: 2.22s remaining: 5.93s 272: learn: 0.2686030 total: 2.22s remaining: 5.92s 273: learn: 0.2683641 total: 2.23s remaining: 5.91s 274: learn: 0.2682184 total: 2.24s remaining: 5.9s 275: learn: 0.2681323 total: 2.25s remaining: 5.89s 276: learn: 0.2679967 total: 2.25s remaining: 5.88s 277: learn: 0.2679201 total: 2.26s remaining: 5.88s 278: learn: 0.2677401 total: 2.27s remaining: 5.87s 279: learn: 0.2674958 total: 2.28s remaining: 5.86s 280: learn: 0.2673855 total: 2.29s remaining: 5.85s 281: learn: 0.2672433 total: 2.29s remaining: 5.84s 282: learn: 0.2671174 total: 2.3s remaining: 5.83s 283: learn: 0.2669682 total: 2.31s remaining: 5.83s 284: learn: 0.2667672 total: 2.32s remaining: 5.82s 285: learn: 0.2666044 total: 2.33s remaining: 5.81s 286: learn: 0.2664981 total: 2.33s remaining: 5.8s 287: learn: 0.2663759 total: 2.34s remaining: 5.79s 288: learn: 0.2662160 total: 2.35s remaining: 5.78s 289: learn: 0.2661149 total: 2.36s remaining: 5.77s 290: learn: 0.2659153 total: 2.36s remaining: 5.76s 291: learn: 0.2657842 total: 2.37s remaining: 5.75s 292: learn: 0.2657279 total: 2.38s remaining: 5.74s 293: learn: 0.2655767 total: 2.39s remaining: 5.73s 294: learn: 0.2654857 total: 2.4s remaining: 5.72s 295: learn: 0.2653991 total: 2.4s remaining: 5.72s 296: learn: 0.2653154 total: 2.41s remaining: 5.71s 297: learn: 0.2652201 total: 2.42s remaining: 5.7s 298: learn: 0.2651340 total: 2.43s remaining: 5.7s 299: learn: 0.2650726 total: 2.44s remaining: 5.69s 300: learn: 0.2648814 total: 2.45s remaining: 5.68s 301: learn: 0.2647603 total: 2.46s remaining: 5.67s 302: learn: 0.2646444 total: 2.46s remaining: 5.67s 303: learn: 0.2645222 total: 2.47s remaining: 5.66s 304: learn: 0.2644040 total: 2.48s remaining: 5.65s 305: learn: 0.2642479 total: 2.49s remaining: 5.64s 306: learn: 0.2641182 total: 2.5s remaining: 5.63s 307: learn: 0.2639839 total: 2.5s remaining: 5.63s 308: learn: 0.2638260 total: 2.51s remaining: 5.62s 309: learn: 0.2637610 total: 2.52s remaining: 5.61s 310: learn: 0.2636361 total: 2.53s remaining: 5.6s 311: learn: 0.2635208 total: 2.54s remaining: 5.59s 312: learn: 0.2633632 total: 2.54s remaining: 5.58s 313: learn: 0.2632738 total: 2.55s remaining: 5.58s 314: learn: 0.2632013 total: 2.56s remaining: 5.57s 315: learn: 0.2630672 total: 2.57s remaining: 5.56s 316: learn: 0.2629070 total: 2.57s remaining: 5.55s 317: learn: 0.2627791 total: 2.58s remaining: 5.54s 318: learn: 0.2626569 total: 2.59s remaining: 5.53s 319: learn: 0.2624693 total: 2.6s remaining: 5.52s 320: learn: 0.2623654 total: 2.6s remaining: 5.51s 321: learn: 0.2621780 total: 2.61s remaining: 5.5s 322: learn: 0.2620440 total: 2.62s remaining: 5.49s 323: learn: 0.2619203 total: 2.63s remaining: 5.48s 324: learn: 0.2617852 total: 2.64s remaining: 5.47s 325: learn: 0.2616589 total: 2.64s remaining: 5.47s 326: learn: 0.2615293 total: 2.65s remaining: 5.46s 327: learn: 0.2614303 total: 2.66s remaining: 5.45s 328: learn: 0.2613447 total: 2.67s remaining: 5.44s 329: learn: 0.2612439 total: 2.68s remaining: 5.44s 330: learn: 0.2611746 total: 2.69s remaining: 5.43s 331: learn: 0.2610849 total: 2.69s remaining: 5.42s 332: learn: 0.2609968 total: 2.7s remaining: 5.41s 333: learn: 0.2608563 total: 2.71s remaining: 5.4s 334: learn: 0.2607413 total: 2.72s remaining: 5.39s 335: learn: 0.2605412 total: 2.73s remaining: 5.38s 336: learn: 0.2604597 total: 2.73s remaining: 5.38s 337: learn: 0.2603115 total: 2.74s remaining: 5.37s 338: learn: 0.2602584 total: 2.75s remaining: 5.36s 339: learn: 0.2601805 total: 2.76s remaining: 5.35s 340: learn: 0.2600239 total: 2.76s remaining: 5.34s 341: learn: 0.2599117 total: 2.77s remaining: 5.33s 342: learn: 0.2597976 total: 2.78s remaining: 5.33s 343: learn: 0.2596856 total: 2.79s remaining: 5.32s 344: learn: 0.2594723 total: 2.79s remaining: 5.31s 345: learn: 0.2593120 total: 2.8s remaining: 5.3s 346: learn: 0.2591039 total: 2.81s remaining: 5.29s 347: learn: 0.2589492 total: 2.82s remaining: 5.28s 348: learn: 0.2587373 total: 2.83s remaining: 5.27s 349: learn: 0.2585984 total: 2.83s remaining: 5.26s 350: learn: 0.2584271 total: 2.84s remaining: 5.25s 351: learn: 0.2582818 total: 2.85s remaining: 5.25s 352: learn: 0.2581235 total: 2.86s remaining: 5.24s 353: learn: 0.2579898 total: 2.87s remaining: 5.23s 354: learn: 0.2578892 total: 2.88s remaining: 5.22s 355: learn: 0.2577189 total: 2.88s remaining: 5.22s 356: learn: 0.2575068 total: 2.89s remaining: 5.21s 357: learn: 0.2573273 total: 2.9s remaining: 5.2s 358: learn: 0.2572316 total: 2.91s remaining: 5.19s 359: learn: 0.2571339 total: 2.92s remaining: 5.18s 360: learn: 0.2570391 total: 2.92s remaining: 5.17s 361: learn: 0.2569610 total: 2.93s remaining: 5.17s 362: learn: 0.2568010 total: 2.94s remaining: 5.16s 363: learn: 0.2565908 total: 2.95s remaining: 5.15s 364: learn: 0.2564256 total: 2.96s remaining: 5.14s 365: learn: 0.2562727 total: 2.96s remaining: 5.13s 366: learn: 0.2561406 total: 2.97s remaining: 5.13s 367: learn: 0.2560296 total: 2.98s remaining: 5.12s 368: learn: 0.2559550 total: 3s remaining: 5.13s 369: learn: 0.2558184 total: 3.01s remaining: 5.12s 370: learn: 0.2556781 total: 3.02s remaining: 5.11s 371: learn: 0.2555820 total: 3.02s remaining: 5.11s 372: learn: 0.2555244 total: 3.03s remaining: 5.1s 373: learn: 0.2553953 total: 3.04s remaining: 5.09s 374: learn: 0.2552735 total: 3.05s remaining: 5.08s 375: learn: 0.2551532 total: 3.06s remaining: 5.07s 376: learn: 0.2550259 total: 3.06s remaining: 5.07s 377: learn: 0.2548621 total: 3.07s remaining: 5.06s 378: learn: 0.2547441 total: 3.08s remaining: 5.05s 379: learn: 0.2545757 total: 3.09s remaining: 5.04s 380: learn: 0.2544704 total: 3.1s remaining: 5.03s 381: learn: 0.2543879 total: 3.1s remaining: 5.02s 382: learn: 0.2542961 total: 3.11s remaining: 5.01s 383: learn: 0.2542287 total: 3.12s remaining: 5s 384: learn: 0.2540925 total: 3.13s remaining: 5s 385: learn: 0.2539513 total: 3.13s remaining: 4.99s 386: learn: 0.2538085 total: 3.14s remaining: 4.98s 387: learn: 0.2536336 total: 3.15s remaining: 4.97s 388: learn: 0.2535215 total: 3.16s remaining: 4.96s 389: learn: 0.2534249 total: 3.17s remaining: 4.95s 390: learn: 0.2533198 total: 3.17s remaining: 4.94s 391: learn: 0.2532631 total: 3.18s remaining: 4.93s 392: learn: 0.2531830 total: 3.19s remaining: 4.92s 393: learn: 0.2530726 total: 3.19s remaining: 4.92s 394: learn: 0.2529418 total: 3.2s remaining: 4.91s 395: learn: 0.2528359 total: 3.21s remaining: 4.9s 396: learn: 0.2526901 total: 3.22s remaining: 4.89s 397: learn: 0.2525773 total: 3.23s remaining: 4.88s 398: learn: 0.2524341 total: 3.24s remaining: 4.88s 399: learn: 0.2522729 total: 3.25s remaining: 4.87s 400: learn: 0.2520926 total: 3.25s remaining: 4.86s 401: learn: 0.2519695 total: 3.26s remaining: 4.85s 402: learn: 0.2518781 total: 3.27s remaining: 4.84s 403: learn: 0.2517732 total: 3.28s remaining: 4.83s 404: learn: 0.2516661 total: 3.28s remaining: 4.82s 405: learn: 0.2515361 total: 3.29s remaining: 4.82s 406: learn: 0.2514157 total: 3.3s remaining: 4.81s 407: learn: 0.2513393 total: 3.31s remaining: 4.8s 408: learn: 0.2512702 total: 3.31s remaining: 4.79s 409: learn: 0.2511839 total: 3.32s remaining: 4.78s 410: learn: 0.2510706 total: 3.33s remaining: 4.77s 411: learn: 0.2509466 total: 3.34s remaining: 4.76s 412: learn: 0.2508211 total: 3.35s remaining: 4.75s 413: learn: 0.2506888 total: 3.35s remaining: 4.75s 414: learn: 0.2505854 total: 3.36s remaining: 4.74s 415: learn: 0.2504620 total: 3.37s remaining: 4.73s 416: learn: 0.2503664 total: 3.38s remaining: 4.72s 417: learn: 0.2502433 total: 3.38s remaining: 4.71s 418: learn: 0.2501237 total: 3.39s remaining: 4.7s 419: learn: 0.2500042 total: 3.4s remaining: 4.7s 420: learn: 0.2498734 total: 3.41s remaining: 4.69s 421: learn: 0.2497835 total: 3.42s remaining: 4.68s 422: learn: 0.2496922 total: 3.43s remaining: 4.67s 423: learn: 0.2495876 total: 3.43s remaining: 4.66s 424: learn: 0.2495088 total: 3.44s remaining: 4.66s 425: learn: 0.2494215 total: 3.45s remaining: 4.65s 426: learn: 0.2493497 total: 3.46s remaining: 4.64s 427: learn: 0.2492174 total: 3.47s remaining: 4.63s 428: learn: 0.2491344 total: 3.47s remaining: 4.62s 429: learn: 0.2490369 total: 3.48s remaining: 4.62s 430: learn: 0.2489147 total: 3.49s remaining: 4.61s 431: learn: 0.2487584 total: 3.5s remaining: 4.6s 432: learn: 0.2486465 total: 3.51s remaining: 4.59s 433: learn: 0.2485167 total: 3.51s remaining: 4.58s 434: learn: 0.2484209 total: 3.52s remaining: 4.57s 435: learn: 0.2483421 total: 3.53s remaining: 4.57s 436: learn: 0.2482036 total: 3.54s remaining: 4.56s 437: learn: 0.2480811 total: 3.54s remaining: 4.55s 438: learn: 0.2479224 total: 3.55s remaining: 4.54s 439: learn: 0.2478263 total: 3.56s remaining: 4.53s 440: learn: 0.2477593 total: 3.57s remaining: 4.52s 441: learn: 0.2476440 total: 3.57s remaining: 4.51s 442: learn: 0.2475507 total: 3.58s remaining: 4.5s 443: learn: 0.2474416 total: 3.59s remaining: 4.5s 444: learn: 0.2473228 total: 3.6s remaining: 4.49s 445: learn: 0.2472174 total: 3.61s remaining: 4.48s 446: learn: 0.2471051 total: 3.61s remaining: 4.47s 447: learn: 0.2470036 total: 3.62s remaining: 4.46s 448: learn: 0.2468908 total: 3.63s remaining: 4.46s 449: learn: 0.2467059 total: 3.64s remaining: 4.45s 450: learn: 0.2466150 total: 3.65s remaining: 4.44s 451: learn: 0.2464962 total: 3.65s remaining: 4.43s 452: learn: 0.2464163 total: 3.66s remaining: 4.42s 453: learn: 0.2462692 total: 3.67s remaining: 4.41s 454: learn: 0.2461763 total: 3.68s remaining: 4.4s 455: learn: 0.2460744 total: 3.68s remaining: 4.39s 456: learn: 0.2459618 total: 3.69s remaining: 4.39s 457: learn: 0.2458762 total: 3.7s remaining: 4.38s 458: learn: 0.2457692 total: 3.71s remaining: 4.37s 459: learn: 0.2456518 total: 3.71s remaining: 4.36s 460: learn: 0.2455437 total: 3.72s remaining: 4.35s 461: learn: 0.2454920 total: 3.73s remaining: 4.34s 462: learn: 0.2453862 total: 3.74s remaining: 4.33s 463: learn: 0.2452353 total: 3.75s remaining: 4.33s 464: learn: 0.2450947 total: 3.75s remaining: 4.32s 465: learn: 0.2449827 total: 3.76s remaining: 4.31s 466: learn: 0.2448741 total: 3.77s remaining: 4.3s 467: learn: 0.2447763 total: 3.78s remaining: 4.29s 468: learn: 0.2446784 total: 3.79s remaining: 4.29s 469: learn: 0.2446025 total: 3.79s remaining: 4.28s 470: learn: 0.2444371 total: 3.8s remaining: 4.27s 471: learn: 0.2443171 total: 3.81s remaining: 4.26s 472: learn: 0.2442051 total: 3.82s remaining: 4.25s 473: learn: 0.2441010 total: 3.82s remaining: 4.24s 474: learn: 0.2439982 total: 3.83s remaining: 4.24s 475: learn: 0.2439315 total: 3.84s remaining: 4.23s 476: learn: 0.2438278 total: 3.85s remaining: 4.22s 477: learn: 0.2437019 total: 3.85s remaining: 4.21s 478: learn: 0.2436112 total: 3.86s remaining: 4.2s 479: learn: 0.2435340 total: 3.87s remaining: 4.19s 480: learn: 0.2434416 total: 3.88s remaining: 4.18s 481: learn: 0.2433807 total: 3.89s remaining: 4.18s 482: learn: 0.2432707 total: 3.89s remaining: 4.17s 483: learn: 0.2431799 total: 3.9s remaining: 4.16s 484: learn: 0.2430503 total: 3.91s remaining: 4.15s 485: learn: 0.2429439 total: 3.92s remaining: 4.14s 486: learn: 0.2428296 total: 3.93s remaining: 4.14s 487: learn: 0.2427242 total: 3.93s remaining: 4.13s 488: learn: 0.2426635 total: 3.94s remaining: 4.12s 489: learn: 0.2426078 total: 3.95s remaining: 4.11s 490: learn: 0.2425378 total: 3.96s remaining: 4.1s 491: learn: 0.2424272 total: 3.97s remaining: 4.09s 492: learn: 0.2423464 total: 3.97s remaining: 4.09s 493: learn: 0.2422399 total: 3.98s remaining: 4.08s 494: learn: 0.2421799 total: 4s remaining: 4.08s 495: learn: 0.2420730 total: 4.01s remaining: 4.08s 496: learn: 0.2420028 total: 4.02s remaining: 4.07s 497: learn: 0.2418598 total: 4.03s remaining: 4.06s 498: learn: 0.2417149 total: 4.03s remaining: 4.05s 499: learn: 0.2415966 total: 4.04s remaining: 4.04s 500: learn: 0.2414307 total: 4.05s remaining: 4.03s 501: learn: 0.2413622 total: 4.06s remaining: 4.03s 502: learn: 0.2412056 total: 4.07s remaining: 4.02s 503: learn: 0.2410850 total: 4.07s remaining: 4.01s 504: learn: 0.2410126 total: 4.08s remaining: 4s 505: learn: 0.2409193 total: 4.09s remaining: 3.99s 506: learn: 0.2407779 total: 4.09s remaining: 3.98s 507: learn: 0.2406883 total: 4.1s remaining: 3.97s 508: learn: 0.2406144 total: 4.11s remaining: 3.96s 509: learn: 0.2405453 total: 4.12s remaining: 3.96s 510: learn: 0.2404668 total: 4.13s remaining: 3.95s 511: learn: 0.2403382 total: 4.13s remaining: 3.94s 512: learn: 0.2402184 total: 4.14s remaining: 3.93s 513: learn: 0.2401061 total: 4.15s remaining: 3.92s 514: learn: 0.2399639 total: 4.16s remaining: 3.92s 515: learn: 0.2398331 total: 4.17s remaining: 3.91s 516: learn: 0.2397196 total: 4.17s remaining: 3.9s 517: learn: 0.2396206 total: 4.18s remaining: 3.89s 518: learn: 0.2395258 total: 4.19s remaining: 3.88s 519: learn: 0.2394617 total: 4.2s remaining: 3.87s 520: learn: 0.2393255 total: 4.21s remaining: 3.87s 521: learn: 0.2392424 total: 4.21s remaining: 3.86s 522: learn: 0.2391117 total: 4.22s remaining: 3.85s 523: learn: 0.2390098 total: 4.23s remaining: 3.84s 524: learn: 0.2389095 total: 4.24s remaining: 3.83s 525: learn: 0.2387522 total: 4.24s remaining: 3.82s 526: learn: 0.2386460 total: 4.25s remaining: 3.82s 527: learn: 0.2385157 total: 4.26s remaining: 3.81s 528: learn: 0.2384005 total: 4.27s remaining: 3.8s 529: learn: 0.2383404 total: 4.27s remaining: 3.79s 530: learn: 0.2382676 total: 4.28s remaining: 3.78s 531: learn: 0.2381505 total: 4.29s remaining: 3.77s 532: learn: 0.2380426 total: 4.3s remaining: 3.77s 533: learn: 0.2379516 total: 4.3s remaining: 3.76s 534: learn: 0.2378672 total: 4.31s remaining: 3.75s 535: learn: 0.2377842 total: 4.32s remaining: 3.74s 536: learn: 0.2377168 total: 4.33s remaining: 3.73s 537: learn: 0.2376473 total: 4.34s remaining: 3.72s 538: learn: 0.2375478 total: 4.34s remaining: 3.72s 539: learn: 0.2374676 total: 4.35s remaining: 3.71s 540: learn: 0.2373656 total: 4.36s remaining: 3.7s 541: learn: 0.2372403 total: 4.37s remaining: 3.69s 542: learn: 0.2371807 total: 4.38s remaining: 3.68s 543: learn: 0.2371210 total: 4.38s remaining: 3.67s 544: learn: 0.2370323 total: 4.39s remaining: 3.67s 545: learn: 0.2369225 total: 4.4s remaining: 3.66s 546: learn: 0.2368419 total: 4.41s remaining: 3.65s 547: learn: 0.2367849 total: 4.42s remaining: 3.64s 548: learn: 0.2366631 total: 4.43s remaining: 3.63s 549: learn: 0.2365313 total: 4.43s remaining: 3.63s 550: learn: 0.2364329 total: 4.44s remaining: 3.62s 551: learn: 0.2363669 total: 4.45s remaining: 3.61s 552: learn: 0.2362535 total: 4.46s remaining: 3.6s 553: learn: 0.2361556 total: 4.47s remaining: 3.6s 554: learn: 0.2360257 total: 4.47s remaining: 3.59s 555: learn: 0.2358791 total: 4.48s remaining: 3.58s 556: learn: 0.2357994 total: 4.49s remaining: 3.57s 557: learn: 0.2357139 total: 4.5s remaining: 3.56s 558: learn: 0.2355981 total: 4.51s remaining: 3.56s 559: learn: 0.2354827 total: 4.51s remaining: 3.55s 560: learn: 0.2353845 total: 4.52s remaining: 3.54s 561: learn: 0.2353248 total: 4.53s remaining: 3.53s 562: learn: 0.2352421 total: 4.54s remaining: 3.52s 563: learn: 0.2351725 total: 4.55s remaining: 3.51s 564: learn: 0.2351462 total: 4.55s remaining: 3.51s 565: learn: 0.2349939 total: 4.56s remaining: 3.5s 566: learn: 0.2349042 total: 4.57s remaining: 3.49s 567: learn: 0.2348001 total: 4.58s remaining: 3.48s 568: learn: 0.2347036 total: 4.58s remaining: 3.47s 569: learn: 0.2346163 total: 4.59s remaining: 3.46s 570: learn: 0.2345250 total: 4.6s remaining: 3.46s 571: learn: 0.2344409 total: 4.61s remaining: 3.45s 572: learn: 0.2343353 total: 4.62s remaining: 3.44s 573: learn: 0.2342590 total: 4.62s remaining: 3.43s 574: learn: 0.2341472 total: 4.63s remaining: 3.42s 575: learn: 0.2340437 total: 4.64s remaining: 3.41s 576: learn: 0.2339859 total: 4.64s remaining: 3.4s 577: learn: 0.2338970 total: 4.65s remaining: 3.4s 578: learn: 0.2338047 total: 4.66s remaining: 3.39s 579: learn: 0.2337383 total: 4.67s remaining: 3.38s 580: learn: 0.2336567 total: 4.68s remaining: 3.37s 581: learn: 0.2335831 total: 4.68s remaining: 3.36s 582: learn: 0.2334758 total: 4.69s remaining: 3.35s 583: learn: 0.2333985 total: 4.7s remaining: 3.35s 584: learn: 0.2332614 total: 4.71s remaining: 3.34s 585: learn: 0.2331571 total: 4.72s remaining: 3.33s 586: learn: 0.2330731 total: 4.72s remaining: 3.32s 587: learn: 0.2329887 total: 4.73s remaining: 3.32s 588: learn: 0.2328950 total: 4.74s remaining: 3.31s 589: learn: 0.2327972 total: 4.75s remaining: 3.3s 590: learn: 0.2327096 total: 4.76s remaining: 3.29s 591: learn: 0.2326265 total: 4.76s remaining: 3.28s 592: learn: 0.2325219 total: 4.77s remaining: 3.27s 593: learn: 0.2324530 total: 4.78s remaining: 3.27s 594: learn: 0.2323814 total: 4.79s remaining: 3.26s 595: learn: 0.2323273 total: 4.79s remaining: 3.25s 596: learn: 0.2322729 total: 4.8s remaining: 3.24s 597: learn: 0.2322046 total: 4.81s remaining: 3.23s 598: learn: 0.2321135 total: 4.82s remaining: 3.23s 599: learn: 0.2319944 total: 4.83s remaining: 3.22s 600: learn: 0.2319582 total: 4.83s remaining: 3.21s 601: learn: 0.2318835 total: 4.84s remaining: 3.2s 602: learn: 0.2318320 total: 4.85s remaining: 3.19s 603: learn: 0.2317575 total: 4.86s remaining: 3.18s 604: learn: 0.2316663 total: 4.87s remaining: 3.18s 605: learn: 0.2315737 total: 4.87s remaining: 3.17s 606: learn: 0.2314896 total: 4.88s remaining: 3.16s 607: learn: 0.2313679 total: 4.89s remaining: 3.15s 608: learn: 0.2312826 total: 4.9s remaining: 3.14s 609: learn: 0.2311869 total: 4.91s remaining: 3.14s 610: learn: 0.2310930 total: 4.91s remaining: 3.13s 611: learn: 0.2310152 total: 4.92s remaining: 3.12s 612: learn: 0.2308936 total: 4.93s remaining: 3.11s 613: learn: 0.2308012 total: 4.94s remaining: 3.1s 614: learn: 0.2306819 total: 4.95s remaining: 3.1s 615: learn: 0.2305891 total: 4.95s remaining: 3.09s 616: learn: 0.2304927 total: 4.96s remaining: 3.08s 617: learn: 0.2304434 total: 4.97s remaining: 3.07s 618: learn: 0.2303537 total: 4.98s remaining: 3.06s 619: learn: 0.2302542 total: 4.99s remaining: 3.06s 620: learn: 0.2301690 total: 5s remaining: 3.05s 621: learn: 0.2300825 total: 5.03s remaining: 3.06s 622: learn: 0.2299744 total: 5.04s remaining: 3.05s 623: learn: 0.2298453 total: 5.05s remaining: 3.04s 624: learn: 0.2297229 total: 5.06s remaining: 3.03s 625: learn: 0.2296675 total: 5.07s remaining: 3.03s 626: learn: 0.2296442 total: 5.07s remaining: 3.02s 627: learn: 0.2295763 total: 5.08s remaining: 3.01s 628: learn: 0.2294868 total: 5.09s remaining: 3s 629: learn: 0.2294090 total: 5.1s remaining: 2.99s 630: learn: 0.2293588 total: 5.11s remaining: 2.98s 631: learn: 0.2292471 total: 5.11s remaining: 2.98s 632: learn: 0.2291806 total: 5.12s remaining: 2.97s 633: learn: 0.2290827 total: 5.13s remaining: 2.96s 634: learn: 0.2290012 total: 5.14s remaining: 2.95s 635: learn: 0.2288757 total: 5.15s remaining: 2.95s 636: learn: 0.2288183 total: 5.16s remaining: 2.94s 637: learn: 0.2286849 total: 5.17s remaining: 2.93s 638: learn: 0.2285324 total: 5.17s remaining: 2.92s 639: learn: 0.2283907 total: 5.2s remaining: 2.92s 640: learn: 0.2283175 total: 5.21s remaining: 2.92s 641: learn: 0.2282168 total: 5.22s remaining: 2.91s 642: learn: 0.2281285 total: 5.22s remaining: 2.9s 643: learn: 0.2280420 total: 5.23s remaining: 2.89s 644: learn: 0.2279308 total: 5.24s remaining: 2.88s 645: learn: 0.2278534 total: 5.25s remaining: 2.88s 646: learn: 0.2278130 total: 5.26s remaining: 2.87s 647: learn: 0.2277102 total: 5.26s remaining: 2.86s 648: learn: 0.2276083 total: 5.27s remaining: 2.85s 649: learn: 0.2275155 total: 5.28s remaining: 2.84s 650: learn: 0.2274131 total: 5.29s remaining: 2.83s 651: learn: 0.2273420 total: 5.3s remaining: 2.83s 652: learn: 0.2272355 total: 5.3s remaining: 2.82s 653: learn: 0.2271202 total: 5.31s remaining: 2.81s 654: learn: 0.2270243 total: 5.32s remaining: 2.8s 655: learn: 0.2268846 total: 5.33s remaining: 2.79s 656: learn: 0.2267749 total: 5.34s remaining: 2.79s 657: learn: 0.2266967 total: 5.35s remaining: 2.78s 658: learn: 0.2265927 total: 5.35s remaining: 2.77s 659: learn: 0.2265394 total: 5.36s remaining: 2.76s 660: learn: 0.2264660 total: 5.37s remaining: 2.75s 661: learn: 0.2263446 total: 5.38s remaining: 2.75s 662: learn: 0.2262680 total: 5.38s remaining: 2.74s 663: learn: 0.2261824 total: 5.39s remaining: 2.73s 664: learn: 0.2261097 total: 5.4s remaining: 2.72s 665: learn: 0.2260224 total: 5.41s remaining: 2.71s 666: learn: 0.2259716 total: 5.42s remaining: 2.7s 667: learn: 0.2259430 total: 5.42s remaining: 2.7s 668: learn: 0.2257905 total: 5.43s remaining: 2.69s 669: learn: 0.2256938 total: 5.44s remaining: 2.68s 670: learn: 0.2256219 total: 5.45s remaining: 2.67s 671: learn: 0.2255524 total: 5.46s remaining: 2.66s 672: learn: 0.2254715 total: 5.46s remaining: 2.65s 673: learn: 0.2254158 total: 5.47s remaining: 2.65s 674: learn: 0.2253839 total: 5.48s remaining: 2.64s 675: learn: 0.2253013 total: 5.49s remaining: 2.63s 676: learn: 0.2251489 total: 5.5s remaining: 2.62s 677: learn: 0.2250699 total: 5.51s remaining: 2.62s 678: learn: 0.2250097 total: 5.51s remaining: 2.61s 679: learn: 0.2248665 total: 5.52s remaining: 2.6s 680: learn: 0.2247965 total: 5.53s remaining: 2.59s 681: learn: 0.2247411 total: 5.54s remaining: 2.58s 682: learn: 0.2246825 total: 5.55s remaining: 2.58s 683: learn: 0.2245700 total: 5.56s remaining: 2.57s 684: learn: 0.2245163 total: 5.56s remaining: 2.56s 685: learn: 0.2244668 total: 5.57s remaining: 2.55s 686: learn: 0.2243382 total: 5.58s remaining: 2.54s 687: learn: 0.2242659 total: 5.59s remaining: 2.53s 688: learn: 0.2241645 total: 5.6s remaining: 2.53s 689: learn: 0.2240620 total: 5.61s remaining: 2.52s 690: learn: 0.2239530 total: 5.62s remaining: 2.51s 691: learn: 0.2238558 total: 5.62s remaining: 2.5s 692: learn: 0.2238156 total: 5.63s remaining: 2.49s 693: learn: 0.2237333 total: 5.64s remaining: 2.49s 694: learn: 0.2236607 total: 5.65s remaining: 2.48s 695: learn: 0.2235790 total: 5.66s remaining: 2.47s 696: learn: 0.2234953 total: 5.66s remaining: 2.46s 697: learn: 0.2234194 total: 5.67s remaining: 2.45s 698: learn: 0.2233113 total: 5.68s remaining: 2.45s 699: learn: 0.2231717 total: 5.69s remaining: 2.44s 700: learn: 0.2230979 total: 5.7s remaining: 2.43s 701: learn: 0.2230280 total: 5.7s remaining: 2.42s 702: learn: 0.2229402 total: 5.71s remaining: 2.41s 703: learn: 0.2228697 total: 5.72s remaining: 2.4s 704: learn: 0.2228250 total: 5.73s remaining: 2.4s 705: learn: 0.2227600 total: 5.74s remaining: 2.39s 706: learn: 0.2226959 total: 5.75s remaining: 2.38s 707: learn: 0.2226225 total: 5.75s remaining: 2.37s 708: learn: 0.2225700 total: 5.76s remaining: 2.36s 709: learn: 0.2224808 total: 5.77s remaining: 2.36s 710: learn: 0.2224083 total: 5.78s remaining: 2.35s 711: learn: 0.2223819 total: 5.79s remaining: 2.34s 712: learn: 0.2222774 total: 5.79s remaining: 2.33s 713: learn: 0.2221774 total: 5.8s remaining: 2.32s 714: learn: 0.2221404 total: 5.81s remaining: 2.31s 715: learn: 0.2220737 total: 5.82s remaining: 2.31s 716: learn: 0.2219876 total: 5.83s remaining: 2.3s 717: learn: 0.2218826 total: 5.83s remaining: 2.29s 718: learn: 0.2218162 total: 5.84s remaining: 2.28s 719: learn: 0.2217588 total: 5.85s remaining: 2.27s 720: learn: 0.2216743 total: 5.86s remaining: 2.27s 721: learn: 0.2215722 total: 5.87s remaining: 2.26s 722: learn: 0.2214773 total: 5.87s remaining: 2.25s 723: learn: 0.2214031 total: 5.88s remaining: 2.24s 724: learn: 0.2213533 total: 5.89s remaining: 2.23s 725: learn: 0.2212922 total: 5.9s remaining: 2.23s 726: learn: 0.2212624 total: 5.9s remaining: 2.22s 727: learn: 0.2211598 total: 5.91s remaining: 2.21s 728: learn: 0.2210599 total: 5.92s remaining: 2.2s 729: learn: 0.2210023 total: 5.93s remaining: 2.19s 730: learn: 0.2209159 total: 5.94s remaining: 2.18s 731: learn: 0.2208022 total: 5.95s remaining: 2.18s 732: learn: 0.2206707 total: 5.95s remaining: 2.17s 733: learn: 0.2205867 total: 5.96s remaining: 2.16s 734: learn: 0.2205174 total: 5.97s remaining: 2.15s 735: learn: 0.2204280 total: 5.98s remaining: 2.14s 736: learn: 0.2203741 total: 5.99s remaining: 2.14s 737: learn: 0.2203094 total: 5.99s remaining: 2.13s 738: learn: 0.2202555 total: 6s remaining: 2.12s 739: learn: 0.2201774 total: 6.01s remaining: 2.11s 740: learn: 0.2201078 total: 6.04s remaining: 2.11s 741: learn: 0.2200129 total: 6.04s remaining: 2.1s 742: learn: 0.2199291 total: 6.05s remaining: 2.09s 743: learn: 0.2198576 total: 6.06s remaining: 2.08s 744: learn: 0.2198044 total: 6.07s remaining: 2.08s 745: learn: 0.2196979 total: 6.08s remaining: 2.07s 746: learn: 0.2195756 total: 6.08s remaining: 2.06s 747: learn: 0.2194931 total: 6.09s remaining: 2.05s 748: learn: 0.2194525 total: 6.1s remaining: 2.04s 749: learn: 0.2194016 total: 6.11s remaining: 2.04s 750: learn: 0.2193025 total: 6.12s remaining: 2.03s 751: learn: 0.2192046 total: 6.12s remaining: 2.02s 752: learn: 0.2191301 total: 6.13s remaining: 2.01s 753: learn: 0.2190905 total: 6.14s remaining: 2s 754: learn: 0.2189554 total: 6.15s remaining: 1.99s 755: learn: 0.2189033 total: 6.16s remaining: 1.99s 756: learn: 0.2188222 total: 6.16s remaining: 1.98s 757: learn: 0.2187622 total: 6.17s remaining: 1.97s 758: learn: 0.2186578 total: 6.18s remaining: 1.96s 759: learn: 0.2185941 total: 6.19s remaining: 1.95s 760: learn: 0.2184803 total: 6.2s remaining: 1.95s 761: learn: 0.2184085 total: 6.2s remaining: 1.94s 762: learn: 0.2183041 total: 6.21s remaining: 1.93s 763: learn: 0.2181917 total: 6.22s remaining: 1.92s 764: learn: 0.2180963 total: 6.23s remaining: 1.91s 765: learn: 0.2180173 total: 6.24s remaining: 1.9s 766: learn: 0.2179318 total: 6.24s remaining: 1.9s 767: learn: 0.2178655 total: 6.25s remaining: 1.89s 768: learn: 0.2177910 total: 6.26s remaining: 1.88s 769: learn: 0.2176954 total: 6.27s remaining: 1.87s 770: learn: 0.2175535 total: 6.28s remaining: 1.86s 771: learn: 0.2174552 total: 6.28s remaining: 1.85s 772: learn: 0.2173661 total: 6.29s remaining: 1.85s 773: learn: 0.2173317 total: 6.3s remaining: 1.84s 774: learn: 0.2172570 total: 6.31s remaining: 1.83s 775: learn: 0.2171970 total: 6.32s remaining: 1.82s 776: learn: 0.2171292 total: 6.32s remaining: 1.81s 777: learn: 0.2170468 total: 6.33s remaining: 1.81s 778: learn: 0.2169581 total: 6.34s remaining: 1.8s 779: learn: 0.2168664 total: 6.35s remaining: 1.79s 780: learn: 0.2168285 total: 6.36s remaining: 1.78s 781: learn: 0.2167979 total: 6.36s remaining: 1.77s 782: learn: 0.2167352 total: 6.37s remaining: 1.77s 783: learn: 0.2166256 total: 6.38s remaining: 1.76s 784: learn: 0.2165272 total: 6.39s remaining: 1.75s 785: learn: 0.2164232 total: 6.4s remaining: 1.74s 786: learn: 0.2163554 total: 6.41s remaining: 1.73s 787: learn: 0.2163180 total: 6.41s remaining: 1.73s 788: learn: 0.2162515 total: 6.42s remaining: 1.72s 789: learn: 0.2162104 total: 6.43s remaining: 1.71s 790: learn: 0.2161252 total: 6.44s remaining: 1.7s 791: learn: 0.2160557 total: 6.45s remaining: 1.69s 792: learn: 0.2159999 total: 6.45s remaining: 1.68s 793: learn: 0.2159272 total: 6.46s remaining: 1.68s 794: learn: 0.2158322 total: 6.47s remaining: 1.67s 795: learn: 0.2157673 total: 6.48s remaining: 1.66s 796: learn: 0.2156839 total: 6.49s remaining: 1.65s 797: learn: 0.2155619 total: 6.49s remaining: 1.64s 798: learn: 0.2154489 total: 6.5s remaining: 1.64s 799: learn: 0.2153295 total: 6.51s remaining: 1.63s 800: learn: 0.2152584 total: 6.52s remaining: 1.62s 801: learn: 0.2151861 total: 6.53s remaining: 1.61s 802: learn: 0.2150888 total: 6.54s remaining: 1.6s 803: learn: 0.2150363 total: 6.54s remaining: 1.59s 804: learn: 0.2149570 total: 6.55s remaining: 1.59s 805: learn: 0.2149127 total: 6.56s remaining: 1.58s 806: learn: 0.2148487 total: 6.57s remaining: 1.57s 807: learn: 0.2147976 total: 6.58s remaining: 1.56s 808: learn: 0.2147059 total: 6.58s remaining: 1.55s 809: learn: 0.2146223 total: 6.59s remaining: 1.55s 810: learn: 0.2145257 total: 6.6s remaining: 1.54s 811: learn: 0.2144414 total: 6.61s remaining: 1.53s 812: learn: 0.2143711 total: 6.62s remaining: 1.52s 813: learn: 0.2143174 total: 6.62s remaining: 1.51s 814: learn: 0.2142902 total: 6.63s remaining: 1.5s 815: learn: 0.2141837 total: 6.64s remaining: 1.5s 816: learn: 0.2141463 total: 6.65s remaining: 1.49s 817: learn: 0.2140290 total: 6.66s remaining: 1.48s 818: learn: 0.2139203 total: 6.67s remaining: 1.47s 819: learn: 0.2138581 total: 6.67s remaining: 1.46s 820: learn: 0.2137948 total: 6.68s remaining: 1.46s 821: learn: 0.2137465 total: 6.69s remaining: 1.45s 822: learn: 0.2136808 total: 6.7s remaining: 1.44s 823: learn: 0.2135773 total: 6.71s remaining: 1.43s 824: learn: 0.2135079 total: 6.71s remaining: 1.42s 825: learn: 0.2134742 total: 6.72s remaining: 1.42s 826: learn: 0.2133768 total: 6.73s remaining: 1.41s 827: learn: 0.2132408 total: 6.74s remaining: 1.4s 828: learn: 0.2131785 total: 6.75s remaining: 1.39s 829: learn: 0.2130925 total: 6.75s remaining: 1.38s 830: learn: 0.2130297 total: 6.76s remaining: 1.38s 831: learn: 0.2129353 total: 6.77s remaining: 1.37s 832: learn: 0.2128608 total: 6.78s remaining: 1.36s 833: learn: 0.2128007 total: 6.78s remaining: 1.35s 834: learn: 0.2127317 total: 6.79s remaining: 1.34s 835: learn: 0.2126471 total: 6.8s remaining: 1.33s 836: learn: 0.2125967 total: 6.81s remaining: 1.33s 837: learn: 0.2125334 total: 6.82s remaining: 1.32s 838: learn: 0.2124508 total: 6.82s remaining: 1.31s 839: learn: 0.2123734 total: 6.83s remaining: 1.3s 840: learn: 0.2123506 total: 6.84s remaining: 1.29s 841: learn: 0.2122618 total: 6.85s remaining: 1.28s 842: learn: 0.2121757 total: 6.86s remaining: 1.28s 843: learn: 0.2120804 total: 6.87s remaining: 1.27s 844: learn: 0.2119854 total: 6.87s remaining: 1.26s 845: learn: 0.2119050 total: 6.88s remaining: 1.25s 846: learn: 0.2118468 total: 6.89s remaining: 1.24s 847: learn: 0.2117937 total: 6.9s remaining: 1.24s 848: learn: 0.2117019 total: 6.91s remaining: 1.23s 849: learn: 0.2116323 total: 6.91s remaining: 1.22s 850: learn: 0.2115975 total: 6.92s remaining: 1.21s 851: learn: 0.2115077 total: 6.93s remaining: 1.2s 852: learn: 0.2114678 total: 6.94s remaining: 1.2s 853: learn: 0.2114577 total: 6.95s remaining: 1.19s 854: learn: 0.2114216 total: 6.95s remaining: 1.18s 855: learn: 0.2113342 total: 6.96s remaining: 1.17s 856: learn: 0.2112600 total: 6.97s remaining: 1.16s 857: learn: 0.2112082 total: 6.98s remaining: 1.15s 858: learn: 0.2111481 total: 6.99s remaining: 1.15s 859: learn: 0.2110980 total: 6.99s remaining: 1.14s 860: learn: 0.2110289 total: 7s remaining: 1.13s 861: learn: 0.2109409 total: 7.01s remaining: 1.12s 862: learn: 0.2108535 total: 7.02s remaining: 1.11s 863: learn: 0.2107722 total: 7.05s remaining: 1.11s 864: learn: 0.2107075 total: 7.05s remaining: 1.1s 865: learn: 0.2106164 total: 7.06s remaining: 1.09s 866: learn: 0.2105748 total: 7.07s remaining: 1.08s 867: learn: 0.2104988 total: 7.08s remaining: 1.08s 868: learn: 0.2104088 total: 7.09s remaining: 1.07s 869: learn: 0.2103383 total: 7.1s remaining: 1.06s 870: learn: 0.2102361 total: 7.11s remaining: 1.05s 871: learn: 0.2101613 total: 7.11s remaining: 1.04s 872: learn: 0.2101200 total: 7.12s remaining: 1.03s 873: learn: 0.2100064 total: 7.13s remaining: 1.03s 874: learn: 0.2099326 total: 7.14s remaining: 1.02s 875: learn: 0.2098829 total: 7.14s remaining: 1.01s 876: learn: 0.2097791 total: 7.15s remaining: 1s 877: learn: 0.2096928 total: 7.16s remaining: 995ms 878: learn: 0.2096116 total: 7.17s remaining: 987ms 879: learn: 0.2095156 total: 7.18s remaining: 979ms 880: learn: 0.2094534 total: 7.19s remaining: 971ms 881: learn: 0.2094165 total: 7.19s remaining: 963ms 882: learn: 0.2093276 total: 7.2s remaining: 954ms 883: learn: 0.2092722 total: 7.21s remaining: 946ms 884: learn: 0.2092006 total: 7.22s remaining: 938ms 885: learn: 0.2091460 total: 7.23s remaining: 930ms 886: learn: 0.2090897 total: 7.24s remaining: 922ms 887: learn: 0.2090399 total: 7.24s remaining: 914ms 888: learn: 0.2090023 total: 7.25s remaining: 905ms 889: learn: 0.2089218 total: 7.26s remaining: 897ms 890: learn: 0.2087982 total: 7.27s remaining: 889ms 891: learn: 0.2087321 total: 7.28s remaining: 881ms 892: learn: 0.2086852 total: 7.28s remaining: 873ms 893: learn: 0.2085606 total: 7.29s remaining: 865ms 894: learn: 0.2084904 total: 7.3s remaining: 856ms 895: learn: 0.2084076 total: 7.31s remaining: 848ms 896: learn: 0.2083522 total: 7.32s remaining: 840ms 897: learn: 0.2082683 total: 7.32s remaining: 832ms 898: learn: 0.2082327 total: 7.33s remaining: 824ms 899: learn: 0.2081190 total: 7.34s remaining: 816ms 900: learn: 0.2080857 total: 7.35s remaining: 808ms 901: learn: 0.2080346 total: 7.36s remaining: 799ms 902: learn: 0.2079675 total: 7.37s remaining: 791ms 903: learn: 0.2078992 total: 7.37s remaining: 783ms 904: learn: 0.2078654 total: 7.38s remaining: 775ms 905: learn: 0.2078071 total: 7.39s remaining: 767ms 906: learn: 0.2077478 total: 7.4s remaining: 758ms 907: learn: 0.2076774 total: 7.41s remaining: 750ms 908: learn: 0.2075978 total: 7.41s remaining: 742ms 909: learn: 0.2075337 total: 7.42s remaining: 734ms 910: learn: 0.2074656 total: 7.43s remaining: 726ms 911: learn: 0.2074219 total: 7.44s remaining: 718ms 912: learn: 0.2073472 total: 7.44s remaining: 709ms 913: learn: 0.2073021 total: 7.45s remaining: 701ms 914: learn: 0.2071893 total: 7.46s remaining: 693ms 915: learn: 0.2071129 total: 7.47s remaining: 685ms 916: learn: 0.2069858 total: 7.48s remaining: 677ms 917: learn: 0.2068877 total: 7.49s remaining: 669ms 918: learn: 0.2068143 total: 7.49s remaining: 660ms 919: learn: 0.2067553 total: 7.5s remaining: 652ms 920: learn: 0.2067146 total: 7.51s remaining: 644ms 921: learn: 0.2066385 total: 7.52s remaining: 636ms 922: learn: 0.2065674 total: 7.53s remaining: 628ms 923: learn: 0.2064703 total: 7.54s remaining: 620ms 924: learn: 0.2063905 total: 7.54s remaining: 612ms 925: learn: 0.2063215 total: 7.55s remaining: 604ms 926: learn: 0.2062382 total: 7.56s remaining: 595ms 927: learn: 0.2061738 total: 7.57s remaining: 587ms 928: learn: 0.2061088 total: 7.58s remaining: 579ms 929: learn: 0.2060524 total: 7.58s remaining: 571ms 930: learn: 0.2060114 total: 7.59s remaining: 563ms 931: learn: 0.2059788 total: 7.6s remaining: 555ms 932: learn: 0.2058946 total: 7.61s remaining: 547ms 933: learn: 0.2058709 total: 7.62s remaining: 538ms 934: learn: 0.2058046 total: 7.63s remaining: 530ms 935: learn: 0.2057623 total: 7.64s remaining: 522ms 936: learn: 0.2057199 total: 7.64s remaining: 514ms 937: learn: 0.2056364 total: 7.65s remaining: 506ms 938: learn: 0.2055876 total: 7.66s remaining: 498ms 939: learn: 0.2055335 total: 7.67s remaining: 490ms 940: learn: 0.2054965 total: 7.68s remaining: 481ms 941: learn: 0.2054202 total: 7.69s remaining: 473ms 942: learn: 0.2053639 total: 7.69s remaining: 465ms 943: learn: 0.2052773 total: 7.7s remaining: 457ms 944: learn: 0.2051818 total: 7.71s remaining: 449ms 945: learn: 0.2051274 total: 7.72s remaining: 441ms 946: learn: 0.2050498 total: 7.73s remaining: 432ms 947: learn: 0.2049594 total: 7.73s remaining: 424ms 948: learn: 0.2049226 total: 7.74s remaining: 416ms 949: learn: 0.2048216 total: 7.75s remaining: 408ms 950: learn: 0.2047653 total: 7.76s remaining: 400ms 951: learn: 0.2047214 total: 7.77s remaining: 392ms 952: learn: 0.2046157 total: 7.78s remaining: 383ms 953: learn: 0.2045532 total: 7.78s remaining: 375ms 954: learn: 0.2044984 total: 7.79s remaining: 367ms 955: learn: 0.2044390 total: 7.8s remaining: 359ms 956: learn: 0.2043388 total: 7.81s remaining: 351ms 957: learn: 0.2042348 total: 7.82s remaining: 343ms 958: learn: 0.2041639 total: 7.82s remaining: 335ms 959: learn: 0.2041364 total: 7.83s remaining: 326ms 960: learn: 0.2040939 total: 7.84s remaining: 318ms 961: learn: 0.2040458 total: 7.85s remaining: 310ms 962: learn: 0.2040047 total: 7.86s remaining: 302ms 963: learn: 0.2039391 total: 7.86s remaining: 294ms 964: learn: 0.2038321 total: 7.87s remaining: 286ms 965: learn: 0.2037040 total: 7.88s remaining: 277ms 966: learn: 0.2036320 total: 7.89s remaining: 269ms 967: learn: 0.2035491 total: 7.9s remaining: 261ms 968: learn: 0.2034893 total: 7.9s remaining: 253ms 969: learn: 0.2034253 total: 7.91s remaining: 245ms 970: learn: 0.2033788 total: 7.92s remaining: 237ms 971: learn: 0.2033201 total: 7.93s remaining: 228ms 972: learn: 0.2032421 total: 7.94s remaining: 220ms 973: learn: 0.2031651 total: 7.95s remaining: 212ms 974: learn: 0.2030931 total: 7.95s remaining: 204ms 975: learn: 0.2030350 total: 7.96s remaining: 196ms 976: learn: 0.2029410 total: 7.97s remaining: 188ms 977: learn: 0.2028739 total: 7.98s remaining: 179ms 978: learn: 0.2028486 total: 7.99s remaining: 171ms 979: learn: 0.2028286 total: 7.99s remaining: 163ms 980: learn: 0.2027439 total: 8s remaining: 155ms 981: learn: 0.2026671 total: 8.01s remaining: 147ms 982: learn: 0.2026179 total: 8.02s remaining: 139ms 983: learn: 0.2025590 total: 8.03s remaining: 131ms 984: learn: 0.2024847 total: 8.03s remaining: 122ms 985: learn: 0.2024418 total: 8.04s remaining: 114ms 986: learn: 0.2023259 total: 8.08s remaining: 106ms 987: learn: 0.2022827 total: 8.09s remaining: 98.2ms 988: learn: 0.2021969 total: 8.09s remaining: 90ms 989: learn: 0.2020952 total: 8.1s remaining: 81.8ms 990: learn: 0.2020277 total: 8.11s remaining: 73.7ms 991: learn: 0.2019527 total: 8.12s remaining: 65.5ms 992: learn: 0.2018579 total: 8.13s remaining: 57.3ms 993: learn: 0.2018068 total: 8.13s remaining: 49.1ms 994: learn: 0.2017433 total: 8.14s remaining: 40.9ms 995: learn: 0.2016975 total: 8.15s remaining: 32.7ms 996: learn: 0.2016517 total: 8.16s remaining: 24.6ms 997: learn: 0.2015688 total: 8.17s remaining: 16.4ms 998: learn: 0.2014925 total: 8.17s remaining: 8.18ms 999: learn: 0.2014498 total: 8.18s remaining: 0us
#predicting
catboost_predictions = catboost_model.predict(X_test)
metrics = ['Precision', 'Recall', 'F1', 'AUC','Accuracy']
eval_metrics = catboost_model.eval_metrics(train_pool,
metrics=metrics,
plot=True)
MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))
As shown in the charts of our model we have quite the impressive metrics that we will be discussing further on in our evalutation but here is an overvie for now
metrics_df = pd.DataFrame({'model':[],'Precision':[], 'Recall':[], 'F1':[], 'AUC':[],'Accuracy':[]})
catboost_metrics = {
'Model':[],
'Precision':[],
'Recall':[],
'F1':[],
'AUC':[],
'Accuracy':[]
}
catboost_metrics['Model']= 'CatBoost'
metrics = ['Precision', 'Recall', 'F1', 'AUC','Accuracy']
for metric in metrics:
catboost_metrics[metric].append(np.mean(eval_metrics[metric]))
print(str(metric)+": {}".format(np.mean(eval_metrics[metric])))
Precision: 0.9604845086979267 Recall: 0.11233274647887324 F1: 0.18988474995917884 AUC: 0.8355849401675709 Accuracy: 0.9133412127440904
Now let's try building other models to see the overall performance of other models compared to our main model and for that we will be using a variety of classifiers.
We will create a function that will automate the learning and the predictions of all of the models and returns a dataframe containing the evaluation metrics
def model(models,X_train, y_train,X_test,y_test):
metrics = {'Model': ['RandomForestClassifier','XGBClassifier','lgb.LGBMClassifier','SVC','LogisticRegression','SGDClassifier'],
'Precision':[],
'Recall':[],
'F1':[],
'AUC':[],
'Accuracy':[]
}
for mod in models:
model = mod
model.fit(X_train, y_train)
predictions = model.predict(X_test)
metrics['Precision'].append(precision_score(y_test,predictions))
metrics['Recall'].append(recall_score(y_test, predictions))
metrics['F1'].append(f1_score(y_test, predictions))
metrics['AUC'].append(roc_auc_score(y_test, predictions))
metrics['Accuracy'].append(accuracy_score(y_test, predictions))
metrics = pd.DataFrame(metrics)
return metrics
models = [RandomForestClassifier(n_estimators=1000),XGBClassifier(),lgb.LGBMClassifier(learning_rate=0.09,max_depth=-5,random_state=42),SVC(),LogisticRegression(),SGDClassifier()]
metrics = model(models,X_train, y_train,X_test,y_test)
metrics_df = metrics.append(pd.DataFrame(catboost_metrics), ignore_index=True)
metrics_df.sort_values(by=['Accuracy'],ascending=False)
| Model | Precision | Recall | F1 | AUC | Accuracy | |
|---|---|---|---|---|---|---|
| 6 | CatBoost | 0.960485 | 0.112333 | 0.189885 | 0.835585 | 0.913341 |
| 2 | lgb.LGBMClassifier | 0.000000 | 0.000000 | 0.000000 | 0.500000 | 0.904452 |
| 3 | SVC | 0.000000 | 0.000000 | 0.000000 | 0.500000 | 0.904452 |
| 1 | XGBClassifier | 0.000000 | 0.000000 | 0.000000 | 0.498864 | 0.902397 |
| 0 | RandomForestClassifier | 0.101562 | 0.465950 | 0.166774 | 0.515254 | 0.555137 |
| 4 | LogisticRegression | 0.096712 | 0.537634 | 0.163934 | 0.503577 | 0.476027 |
| 5 | SGDClassifier | 0.095405 | 0.878136 | 0.172111 | 0.499273 | 0.192808 |
CatBoost confusion matrix
from sklearn.metrics import confusion_matrix
cf_matrix = confusion_matrix(y_test, catboost_predictions)
sns.heatmap(cf_matrix/np.sum(cf_matrix), annot=True,
fmt='.2%', cmap='Blues')
<AxesSubplot:>
Let's export our model in order to build a web application and give insights to our client
import pickle
# open a file, where you ant to store the data
file = open('cat_boost_regression_model.pkl', 'wb')
# dump information to that file
pickle.dump(catboost_model, file)